Jul 05 13:36:43.135651 ip-10-0-142-97 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 05 13:36:43.135663 ip-10-0-142-97 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 05 13:36:43.135670 ip-10-0-142-97 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 05 13:36:43.135874 ip-10-0-142-97 systemd[1]: Failed to start Kubernetes Kubelet. Jul 05 13:36:53.357234 ip-10-0-142-97 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 05 13:36:53.357248 ip-10-0-142-97 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot fcf774f94cf34941a2c5e8e48bbce4ca -- Jul 05 13:38:43.701592 ip-10-0-142-97 systemd[1]: Starting Kubernetes Kubelet... Jul 05 13:38:44.152892 ip-10-0-142-97 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:38:44.152892 ip-10-0-142-97 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 13:38:44.152892 ip-10-0-142-97 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:38:44.152892 ip-10-0-142-97 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 13:38:44.152892 ip-10-0-142-97 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:38:44.844760 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.844550 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852040 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852053 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852057 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852060 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852066 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852070 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852074 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852078 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852091 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852094 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852097 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852100 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852103 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852105 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852108 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852111 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852114 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852117 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852120 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:38:44.986437 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852123 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852126 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852128 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852131 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852134 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852137 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852141 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852146 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852150 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852153 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852157 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852161 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852164 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852167 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852170 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852173 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852176 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852179 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852181 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852191 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:38:44.988532 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852194 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852197 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852199 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852202 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852205 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852207 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852210 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852213 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852215 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852218 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852221 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852224 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852227 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852229 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852232 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852235 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852237 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852240 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852243 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:38:44.989373 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852245 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852248 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852250 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852254 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852258 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852260 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852263 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852266 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852268 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852271 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852274 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852277 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852279 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852294 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852297 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852300 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852303 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852305 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852308 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852311 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:38:44.992395 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852313 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852316 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852320 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852323 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852326 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852328 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852331 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852334 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852337 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852339 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852344 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852348 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852351 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852354 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852357 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852360 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852363 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852366 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852369 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:38:44.993229 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852372 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.852374 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853227 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853233 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853237 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853239 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853242 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853248 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853252 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853255 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853257 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853260 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853263 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853265 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853268 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853271 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853274 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853276 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853279 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853282 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:38:44.996217 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853285 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853288 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853291 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853294 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853296 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853299 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853303 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853305 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853308 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853313 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853318 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853321 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853324 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853327 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853330 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853333 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853336 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853339 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853342 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:38:44.997063 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853344 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853348 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853351 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853354 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853357 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853360 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853363 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853367 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853372 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853376 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853380 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853401 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853404 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853407 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853410 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853413 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853415 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853418 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853421 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:38:45.000481 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853424 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853428 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853431 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853433 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853436 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853439 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853441 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853444 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853447 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853449 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853452 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853455 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853458 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853461 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853463 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853468 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853471 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853473 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853478 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853482 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:38:45.001307 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853484 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853487 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853490 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853493 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853496 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853498 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853503 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853505 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853508 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853511 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853514 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853517 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853519 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853522 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853525 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853528 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853530 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853533 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853535 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853538 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:38:45.004276 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853541 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853543 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853546 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853969 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853974 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853977 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853980 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853984 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853987 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853990 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853994 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.853997 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854000 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854003 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854005 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854008 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854011 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854014 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854018 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854021 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:38:45.005142 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854024 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854027 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854029 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854032 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854035 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854038 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854040 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854043 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854046 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854049 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854051 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854054 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854057 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854060 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854062 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854065 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854067 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854070 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854073 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854075 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:38:45.009445 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854078 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854081 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854085 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854088 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854091 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854093 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854096 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854099 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854101 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854104 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854108 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854110 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854114 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854117 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854119 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854122 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854125 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854128 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854130 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:38:45.010288 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854135 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854139 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854142 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854145 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854148 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854151 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854154 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854157 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854160 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854163 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854166 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854169 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854172 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854175 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854178 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854182 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854185 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854187 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854190 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854193 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:38:45.014519 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854195 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854198 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854200 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854204 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854207 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854210 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854214 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854217 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854219 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854223 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854227 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854230 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854233 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854236 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854239 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854242 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854245 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854248 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854251 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:38:45.015338 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854254 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854256 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854259 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.854261 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854336 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854349 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854362 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854367 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854398 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854403 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854407 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854412 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854415 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854419 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854422 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854426 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854430 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854434 2560 flags.go:64] FLAG: --cgroup-root="" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854437 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854444 2560 flags.go:64] FLAG: --client-ca-file="" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854447 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854450 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854457 2560 flags.go:64] FLAG: --cluster-domain="" Jul 05 13:38:45.018707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854460 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854463 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854466 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854470 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854474 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854477 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854480 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854484 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854488 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854491 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854494 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854497 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854500 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854505 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854508 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854511 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854514 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854519 2560 flags.go:64] FLAG: --enable-server="true" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854528 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854536 2560 flags.go:64] FLAG: --event-burst="100" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854539 2560 flags.go:64] FLAG: --event-qps="50" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854542 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854546 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854549 2560 flags.go:64] FLAG: --eviction-hard="" Jul 05 13:38:45.019963 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854553 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854557 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854561 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854564 2560 flags.go:64] FLAG: --eviction-soft="" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854567 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854570 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854573 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854576 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854580 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854583 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854586 2560 flags.go:64] FLAG: --feature-gates="" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854590 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854593 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854596 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854599 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854603 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854606 2560 flags.go:64] FLAG: --help="false" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854609 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854612 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854615 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854618 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854621 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854625 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854628 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 13:38:45.157066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854631 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 13:38:45.040247 ip-10-0-142-97 systemd[1]: Started Kubernetes Kubelet. Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854635 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854638 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854647 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854651 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854653 2560 flags.go:64] FLAG: --kube-reserved="" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854657 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854660 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854663 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854667 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854670 2560 flags.go:64] FLAG: --lock-file="" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854673 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854676 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854680 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854686 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854689 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854692 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854695 2560 flags.go:64] FLAG: --logging-format="text" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854697 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854701 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854704 2560 flags.go:64] FLAG: --manifest-url="" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854707 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854711 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854714 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854719 2560 flags.go:64] FLAG: --max-pods="110" Jul 05 13:38:45.159040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854722 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854725 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854728 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854731 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854734 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854737 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854740 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854748 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854753 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854756 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854760 2560 flags.go:64] FLAG: --pod-cidr="" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854773 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854779 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854782 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854785 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854788 2560 flags.go:64] FLAG: --port="10250" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854792 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854796 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01bf61c60fa413631" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854800 2560 flags.go:64] FLAG: --qos-reserved="" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854803 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854806 2560 flags.go:64] FLAG: --register-node="true" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854809 2560 flags.go:64] FLAG: --register-with-taints="" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854813 2560 flags.go:64] FLAG: --registry-burst="10" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854816 2560 flags.go:64] FLAG: --registry-qps="5" Jul 05 13:38:45.163772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854819 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854822 2560 flags.go:64] FLAG: --reserved-memory="" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854826 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854829 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854832 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854835 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854838 2560 flags.go:64] FLAG: --runonce="false" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854841 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854844 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854847 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854851 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854854 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854857 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854860 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854863 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854866 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854871 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854874 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854877 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854880 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854889 2560 flags.go:64] FLAG: --system-cgroups="" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854892 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854898 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854901 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854904 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 13:38:45.164600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854912 2560 flags.go:64] FLAG: --tls-min-version="" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854915 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854918 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854921 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854924 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854927 2560 flags.go:64] FLAG: --v="2" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854931 2560 flags.go:64] FLAG: --version="false" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854936 2560 flags.go:64] FLAG: --vmodule="" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854940 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.854943 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855078 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855082 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855085 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855088 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855091 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855096 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855099 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855102 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855105 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855108 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855111 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855113 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:38:45.169103 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855116 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855120 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855123 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855126 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855129 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855133 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855137 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855140 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855143 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855146 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855151 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855154 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855157 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855160 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855163 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855166 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855168 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855171 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855174 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:38:45.169911 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855177 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855180 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855182 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855185 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855188 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855191 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855194 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855196 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855199 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855202 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855205 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855207 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855210 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855212 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855216 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855219 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855222 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855225 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855227 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855230 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:38:45.173190 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855236 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855238 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855241 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855245 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855248 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855251 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855254 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855256 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855259 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855262 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855264 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855267 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855270 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855273 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855276 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855278 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855281 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855285 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855289 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855293 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:38:45.173986 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855295 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855299 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855302 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855305 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855313 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855317 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855321 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855324 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855326 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855329 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855332 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855334 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855337 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855346 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855349 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855354 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855356 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855359 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855362 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855365 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:38:45.176877 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855367 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855370 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855373 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855376 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855378 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855381 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855396 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.855399 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.855405 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.864302 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.864318 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.864340 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.864346 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864428 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864434 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:38:45.177714 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864438 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864441 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864444 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864447 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864450 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864454 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864459 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864464 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864467 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864470 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864474 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864478 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864481 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864483 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864486 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864489 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864492 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864494 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864497 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:38:45.181780 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864500 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864503 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864506 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864508 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864511 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864514 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864517 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864520 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864525 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864529 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864532 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864535 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864538 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864541 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864544 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864548 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864551 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864554 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864557 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864560 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:38:45.182598 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864563 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864566 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864568 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864571 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864575 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864578 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864580 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864583 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864587 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864591 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864594 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864597 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864599 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864602 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864605 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864608 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864611 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864613 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864616 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:38:45.187520 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864620 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864622 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864628 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864631 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864634 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864638 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864640 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864643 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864646 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864649 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864651 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864654 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864657 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864659 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864662 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864665 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864668 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864671 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864674 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864677 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:38:45.188287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864680 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864682 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864685 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864688 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864691 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864694 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864696 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864699 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864702 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864705 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864708 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864711 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864714 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864717 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864721 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864724 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864726 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864729 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:38:45.189104 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864731 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.864737 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864857 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864862 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864865 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864868 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864872 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864875 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864879 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864882 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864885 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864888 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864891 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864894 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864897 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:38:45.292831 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864900 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864903 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864906 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864909 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864912 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864915 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864918 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864920 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864923 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864926 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864928 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864932 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864935 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864938 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864941 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864944 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864947 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864949 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864952 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:38:45.293644 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864955 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864958 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864960 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864963 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864967 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864970 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864973 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864976 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864979 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864982 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864984 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864987 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864992 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864995 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.864998 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865002 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865004 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865007 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865010 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:38:45.294898 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865013 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865016 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865019 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865021 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865024 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865027 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865029 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865033 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865036 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865038 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865041 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865044 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865048 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865051 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865055 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865058 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865060 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865066 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865068 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865071 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:38:45.297421 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865075 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865078 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865080 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865083 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865086 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865089 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865092 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865095 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865097 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865100 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865102 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865105 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865108 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865111 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865114 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865116 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865119 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865121 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865126 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:38:45.298248 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865130 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865134 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865137 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865139 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865142 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865145 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865148 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865151 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:44.865154 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.865159 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.865292 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.867545 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.869194 2560 server.go:1019] "Starting client certificate rotation" Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.869323 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:38:45.301158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.869362 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:38:45.301976 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.873083 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:38:45.301976 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.881893 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:38:45.301976 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.891022 2560 log.go:25] "Validated CRI v1 runtime API" Jul 05 13:38:45.301976 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.893327 2560 log.go:25] "Validated CRI v1 image API" Jul 05 13:38:45.301976 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.894139 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 13:38:45.301976 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.899034 2560 fs.go:135] Filesystem UUIDs: map[5043f475-0769-49c8-b36d-2d00f15cefe2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e27848c1-1cc7-4145-8581-ccc24f05afb7:/dev/nvme0n1p4] Jul 05 13:38:45.301976 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.899054 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.904980 2560 manager.go:216] Machine: {Timestamp:2026-07-05 13:38:44.903403391 +0000 UTC m=+1.083245833 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103979 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29af093642bf627d3a1c4f6660e50c SystemUUID:ec29af09-3642-bf62-7d3a-1c4f6660e50c BootID:fcf774f9-4cf3-4941-a2c5-e8e48bbce4ca Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:68:ae:7c:58:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:68:ae:7c:58:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:31:9a:5f:23:db Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.905090 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.905165 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.908933 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.908965 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-97.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.914725 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.914742 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.914751 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.915475 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.917078 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.921442 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.921579 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.921753 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.921772 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.921787 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.921796 2560 kubelet.go:398] "Adding apiserver pod source" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.921865 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 13:38:45.404577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.922237 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.922257 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.924653 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.925800 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:44.940423 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:44.982641 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ntdwb" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.031593 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ntdwb" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.031674 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-97.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.032995 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.033047 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034115 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034130 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034138 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034156 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034167 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034174 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034183 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034202 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034211 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034217 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034226 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.034233 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.035869 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.035905 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.039254 2560 server.go:1268] "Started kubelet" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.039359 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.039340 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.039471 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.040570 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.041389 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.049482 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.051909 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.051925 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.052026 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.052525 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.052551 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.052569 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.052723 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.052734 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.052952 2560 factory.go:55] Registering systemd factory Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.052967 2560 factory.go:223] Registration of the systemd container factory successfully Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.053195 2560 factory.go:153] Registering CRI-O factory Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.053205 2560 factory.go:223] Registration of the crio container factory successfully Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.053246 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.053301 2560 factory.go:103] Registering Raw factory Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.053313 2560 manager.go:1196] Started watching for new ooms in manager Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.053977 2560 manager.go:349] Starting recovery of all containers Jul 05 13:38:45.413615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.055317 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.058546 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-97.ec2.internal\" not found" node="ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.060775 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.068939 2560 manager.go:354] Recovery completed Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.074648 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.076995 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.077020 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.077030 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.077502 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.077516 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.077536 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.077507 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.078230 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.083932 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.083951 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.083960 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.085048 2560 policy_none.go:49] "None policy: Start" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.085061 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.085073 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.087220 2560 policy_none.go:47] "Start" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.140504 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.152650 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.253043 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.353504 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.401484 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.415670 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.401545 2560 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.440663 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.440646 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.453812 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.453799 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.458067 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.458051 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.515771 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.515758 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-97.ec2.internal" not found Jul 05 13:38:45.533744 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.533731 2560 manager.go:337] "Starting Device Plugin manager" Jul 05 13:38:45.533797 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.533764 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 13:38:45.533797 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.533773 2560 server.go:87] "Starting device plugin registration server" Jul 05 13:38:45.534084 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.534070 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 13:38:45.534139 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.534086 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 13:38:45.534270 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.534176 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 13:38:45.534270 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.534255 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 13:38:45.534270 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.534263 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 13:38:45.534911 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.534888 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 13:38:45.534982 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.534956 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.583563 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.583512 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 13:38:45.585826 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.584776 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 13:38:45.585826 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.584810 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 13:38:45.585826 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.584829 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 13:38:45.585826 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.584859 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 13:38:45.588034 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.588014 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:38:45.634600 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.634581 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:38:45.635618 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.635597 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:38:45.635687 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.635633 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:38:45.635687 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.635646 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:38:45.635687 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.635674 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.643647 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.643627 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.643740 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.643650 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-97.ec2.internal\": node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.661822 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.661803 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.685361 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.685331 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal"] Jul 05 13:38:45.685413 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.685396 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:38:45.687517 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.687502 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:38:45.687609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.687533 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:38:45.687609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.687548 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:38:45.689990 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.689976 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:38:45.690127 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690111 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.690180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690148 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:38:45.690764 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690747 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:38:45.690764 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690775 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:38:45.690898 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690786 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:38:45.690898 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690749 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:38:45.690898 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690867 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:38:45.690898 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.690884 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:38:45.693289 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.693273 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.693400 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.693302 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:38:45.693941 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.693925 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:38:45.694059 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.693953 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:38:45.694059 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.693963 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:38:45.705943 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.705920 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-97.ec2.internal\" not found" node="ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.710647 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.710630 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-97.ec2.internal\" not found" node="ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.757251 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.757228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ff4bc349d7008d7ce7073a381e0e0de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"2ff4bc349d7008d7ce7073a381e0e0de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.757365 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.757259 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2f4209627eb6491cbb2535de8e653eb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-97.ec2.internal\" (UID: \"f2f4209627eb6491cbb2535de8e653eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.757365 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.757285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ff4bc349d7008d7ce7073a381e0e0de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"2ff4bc349d7008d7ce7073a381e0e0de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.762939 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.762924 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.858020 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.857973 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ff4bc349d7008d7ce7073a381e0e0de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"2ff4bc349d7008d7ce7073a381e0e0de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.858020 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.858005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ff4bc349d7008d7ce7073a381e0e0de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"2ff4bc349d7008d7ce7073a381e0e0de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.858118 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.858050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2f4209627eb6491cbb2535de8e653eb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-97.ec2.internal\" (UID: \"f2f4209627eb6491cbb2535de8e653eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.858118 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.858077 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2f4209627eb6491cbb2535de8e653eb-config\") pod \"kube-apiserver-proxy-ip-10-0-142-97.ec2.internal\" (UID: \"f2f4209627eb6491cbb2535de8e653eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.858180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.858123 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ff4bc349d7008d7ce7073a381e0e0de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"2ff4bc349d7008d7ce7073a381e0e0de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.858180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.858144 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ff4bc349d7008d7ce7073a381e0e0de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"2ff4bc349d7008d7ce7073a381e0e0de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:45.863523 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.863507 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:45.868707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.868686 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 13:38:45.868843 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.868823 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 13:38:45.868912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:45.868823 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 13:38:45.964614 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:45.964584 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:46.012359 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.012341 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:46.014684 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.014669 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 05 13:38:46.037948 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.037916 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 13:33:45 +0000 UTC" deadline="2028-04-06 02:16:43.742961202 +0000 UTC" Jul 05 13:38:46.037948 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.037942 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15372h37m57.705021785s" Jul 05 13:38:46.065654 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:46.065628 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:46.125961 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.125910 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:38:46.166523 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:46.166493 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 05 13:38:46.229809 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.229786 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:38:46.252820 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.252786 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 05 13:38:46.269165 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.269145 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:38:46.270230 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.270215 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 05 13:38:46.280786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.280766 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:38:46.579492 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:46.579457 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff4bc349d7008d7ce7073a381e0e0de.slice/crio-b0428313e552efe2475f67e7db0c5f04247cec30593afbd894df1dd605ebc786 WatchSource:0}: Error finding container b0428313e552efe2475f67e7db0c5f04247cec30593afbd894df1dd605ebc786: Status 404 returned error can't find the container with id b0428313e552efe2475f67e7db0c5f04247cec30593afbd894df1dd605ebc786 Jul 05 13:38:46.585775 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.585753 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:38:46.587949 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.587913 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" event={"ID":"2ff4bc349d7008d7ce7073a381e0e0de","Type":"ContainerStarted","Data":"b0428313e552efe2475f67e7db0c5f04247cec30593afbd894df1dd605ebc786"} Jul 05 13:38:46.592400 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:46.592366 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f4209627eb6491cbb2535de8e653eb.slice/crio-83d846e24125733a7fc851ae2e294bde5b3e48e4f607c22c1047dfcacb8b721a WatchSource:0}: Error finding container 83d846e24125733a7fc851ae2e294bde5b3e48e4f607c22c1047dfcacb8b721a: Status 404 returned error can't find the container with id 83d846e24125733a7fc851ae2e294bde5b3e48e4f607c22c1047dfcacb8b721a Jul 05 13:38:46.692483 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.692458 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:38:46.925750 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.925723 2560 apiserver.go:52] "Watching apiserver" Jul 05 13:38:46.937242 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.937217 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 13:38:46.937621 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.937600 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal","openshift-dns/node-resolver-r7c9v","openshift-image-registry/node-ca-6lmkh","openshift-multus/multus-srgqx","openshift-network-operator/iptables-alerter-nx8w8","openshift-ovn-kubernetes/ovnkube-node-dq5h4","kube-system/global-pull-secret-syncer-9wxg4","kube-system/konnectivity-agent-nmbbx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4","openshift-cluster-node-tuning-operator/tuned-r2fpj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal","openshift-multus/multus-additional-cni-plugins-r4ct4","openshift-multus/network-metrics-daemon-n66xq","openshift-network-diagnostics/network-check-target-m4t56"] Jul 05 13:38:46.940527 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.940507 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:46.940616 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:46.940593 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:38:46.942673 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.942657 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:46.942736 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:46.942717 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:38:46.944955 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.944934 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:46.947245 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.947222 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.947719 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.947694 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 13:38:46.947829 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.947736 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 13:38:46.947829 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.947808 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p279v\"" Jul 05 13:38:46.947924 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.947859 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 13:38:46.949559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.949539 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:46.950552 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.950536 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 13:38:46.950714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.950695 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 13:38:46.950789 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.950746 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 13:38:46.950789 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.950750 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vdl8k\"" Jul 05 13:38:46.950889 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.950751 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 13:38:46.952061 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.952042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.952146 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.952107 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 13:38:46.952208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.952170 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:38:46.952262 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.952250 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 13:38:46.952548 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.952531 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vbkcn\"" Jul 05 13:38:46.954569 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.954548 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.954666 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.954583 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 13:38:46.954899 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.954854 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 13:38:46.955000 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.954914 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 13:38:46.955000 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.954922 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 13:38:46.955115 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.955044 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z2k5t\"" Jul 05 13:38:46.955115 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.955058 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 13:38:46.956601 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.956583 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 13:38:46.957154 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.957139 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:46.957235 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:46.957204 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:38:46.959757 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.959222 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zm276\"" Jul 05 13:38:46.959757 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.959237 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 13:38:46.959757 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.959529 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 13:38:46.959757 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.959537 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 13:38:46.961227 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.961204 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:46.963524 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.963504 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:46.963811 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.963785 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wg27n\"" Jul 05 13:38:46.963909 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.963818 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 13:38:46.963909 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.963823 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 13:38:46.964752 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clzl9\" (UniqueName: \"kubernetes.io/projected/c424c3fa-89b7-43e4-80e9-a09466ee2d89-kube-api-access-clzl9\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:46.964818 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.964818 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-cni-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.964818 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-cnibin\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.964947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-cni-multus\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.964947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-dbus\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:46.964947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964885 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-node-log\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.964947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-log-socket\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.964947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx9pz\" (UniqueName: \"kubernetes.io/projected/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-kube-api-access-tx9pz\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:46.964947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964942 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-systemd-units\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-system-cni-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.964980 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd756\" (UniqueName: \"kubernetes.io/projected/165d05a1-21b6-4a1f-9069-da352a4aa357-kube-api-access-sd756\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965014 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-kubelet\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-systemd\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965049 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-run-ovn-kubernetes\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-os-release\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-socket-dir-parent\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-kubelet\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-ovnkube-config\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965156 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-registration-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-etc-selinux\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965237 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-env-overrides\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/165d05a1-21b6-4a1f-9069-da352a4aa357-host-slash\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw82v\" (UniqueName: \"kubernetes.io/projected/6a918c1c-3a68-4e16-aa80-ea202e86d206-kube-api-access-cw82v\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965297 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965360 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-ovnkube-script-lib\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965405 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-cni-bin\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c57b80fc-d60a-4a29-9101-c71955868288-multus-daemon-config\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965459 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-run-netns\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965490 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-ovn\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9759e393-3424-4efd-afac-56eedc206f49-ovn-node-metrics-cert\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965535 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-host\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965550 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:46.965594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965564 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-serviceca\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965585 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-var-lib-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965601 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-cni-bin\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965617 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-conf-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965632 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-etc-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-k8s-cni-cncf-io\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965681 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28g4c\" (UniqueName: \"kubernetes.io/projected/c57b80fc-d60a-4a29-9101-c71955868288-kube-api-access-28g4c\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-slash\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965713 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/165d05a1-21b6-4a1f-9069-da352a4aa357-iptables-alerter-script\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5znfx\" (UniqueName: \"kubernetes.io/projected/9759e393-3424-4efd-afac-56eedc206f49-kube-api-access-5znfx\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965744 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-sys-fs\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965759 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-cni-netd\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57b80fc-d60a-4a29-9101-c71955868288-cni-binary-copy\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965788 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-hostroot\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-multus-certs\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-socket-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.966356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-device-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:46.966865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-kubelet-config\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:46.966865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965899 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xkdhb\"" Jul 05 13:38:46.966865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-netns\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965939 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:46.966865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.965939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-etc-kubernetes\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:46.966865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.966356 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 13:38:46.966865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.966691 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:38:46.968395 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.968366 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:46.968536 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.968518 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2s2l9\"" Jul 05 13:38:46.968604 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.968573 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 13:38:46.968768 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.968751 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 13:38:46.971016 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.970994 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 13:38:46.971138 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.971017 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rv8pc\"" Jul 05 13:38:46.971138 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:46.971046 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 13:38:47.053290 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.053265 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 13:38:47.054376 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.054308 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-etc-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-k8s-cni-cncf-io\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066510 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-etc-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066524 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28g4c\" (UniqueName: \"kubernetes.io/projected/c57b80fc-d60a-4a29-9101-c71955868288-kube-api-access-28g4c\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-cnibin\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066625 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-k8s-cni-cncf-io\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-kubernetes\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.066681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066672 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b533b9df-1afd-48fe-8b93-dff258281d4e-tmp\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-slash\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066737 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/165d05a1-21b6-4a1f-9069-da352a4aa357-iptables-alerter-script\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a36113f7-8e41-48ce-9a66-156e196e4c60-agent-certs\") pod \"konnectivity-agent-nmbbx\" (UID: \"a36113f7-8e41-48ce-9a66-156e196e4c60\") " pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066802 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5znfx\" (UniqueName: \"kubernetes.io/projected/9759e393-3424-4efd-afac-56eedc206f49-kube-api-access-5znfx\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-sys-fs\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066826 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-slash\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066838 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-modprobe-d\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-cni-netd\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57b80fc-d60a-4a29-9101-c71955868288-cni-binary-copy\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-hostroot\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-multus-certs\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.066982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-socket-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-device-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067085 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-kubelet-config\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:47.067350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-netns\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-etc-kubernetes\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-sys\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067213 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-host\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067241 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clzl9\" (UniqueName: \"kubernetes.io/projected/c424c3fa-89b7-43e4-80e9-a09466ee2d89-kube-api-access-clzl9\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067266 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067294 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f48763-233d-4180-9ae8-cd2e5934ef3b-tmp-dir\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067357 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-run\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-cni-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-cnibin\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-cni-multus\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067495 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rzxt\" (UniqueName: \"kubernetes.io/projected/d1f48763-233d-4180-9ae8-cd2e5934ef3b-kube-api-access-4rzxt\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-dbus\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067521 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-netns\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067590 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-etc-kubernetes\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.068345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067799 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-sys-fs\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067851 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-cni-netd\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067884 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067935 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-cni-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067990 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-cnibin\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068034 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-cni-multus\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.067468 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/165d05a1-21b6-4a1f-9069-da352a4aa357-iptables-alerter-script\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068171 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-dbus\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068214 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-device-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068253 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-hostroot\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068294 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-run-multus-certs\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068379 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57b80fc-d60a-4a29-9101-c71955868288-cni-binary-copy\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068424 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-socket-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068472 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-node-log\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068487 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-kubelet-config\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068501 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-log-socket\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068528 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-node-log\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068529 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx9pz\" (UniqueName: \"kubernetes.io/projected/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-kube-api-access-tx9pz\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:47.069367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysconfig\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-var-lib-kubelet\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068630 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-systemd-units\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-system-cni-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068679 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd756\" (UniqueName: \"kubernetes.io/projected/165d05a1-21b6-4a1f-9069-da352a4aa357-kube-api-access-sd756\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068704 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-lib-modules\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-kubelet\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068742 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-log-socket\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-systemd\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-run-ovn-kubernetes\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068800 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-system-cni-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068808 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-os-release\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-socket-dir-parent\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068851 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-systemd-units\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-kubelet\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068886 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v56n\" (UniqueName: \"kubernetes.io/projected/5730bfca-1a7a-449b-8205-5d938dd443ef-kube-api-access-9v56n\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068891 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-systemd\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.071935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068902 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1f48763-233d-4180-9ae8-cd2e5934ef3b-hosts-file\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068925 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-ovnkube-config\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-registration-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.068998 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-etc-selinux\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysctl-conf\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-env-overrides\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/165d05a1-21b6-4a1f-9069-da352a4aa357-host-slash\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069102 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw82v\" (UniqueName: \"kubernetes.io/projected/6a918c1c-3a68-4e16-aa80-ea202e86d206-kube-api-access-cw82v\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069113 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-kubelet\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-os-release\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069184 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-ovnkube-script-lib\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069210 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-cni-bin\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c57b80fc-d60a-4a29-9101-c71955868288-multus-daemon-config\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-tuned\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a36113f7-8e41-48ce-9a66-156e196e4c60-konnectivity-ca\") pod \"konnectivity-agent-nmbbx\" (UID: \"a36113f7-8e41-48ce-9a66-156e196e4c60\") " pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:47.072838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-run-netns\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-ovn\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9759e393-3424-4efd-afac-56eedc206f49-ovn-node-metrics-cert\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-host\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069450 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysctl-d\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069462 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069473 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-systemd\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069604 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-run-netns\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069653 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-etc-selinux\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069655 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069666 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-ovnkube-config\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069693 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a918c1c-3a68-4e16-aa80-ea202e86d206-registration-dir\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069709 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-kubelet\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069745 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-run-ovn-kubernetes\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069769 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-os-release\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.073717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069797 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-socket-dir-parent\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.069836 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/165d05a1-21b6-4a1f-9069-da352a4aa357-host-slash\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-serviceca\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070046 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-env-overrides\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070070 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-system-cni-dir\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070098 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxfmb\" (UniqueName: \"kubernetes.io/projected/b533b9df-1afd-48fe-8b93-dff258281d4e-kube-api-access-vxfmb\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070101 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-host-var-lib-cni-bin\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070094 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070128 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-var-lib-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070133 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-run-ovn\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.070213 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.070279 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:47.570255897 +0000 UTC m=+3.750098326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.070679 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070718 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9759e393-3424-4efd-afac-56eedc206f49-ovnkube-script-lib\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.070732 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret podName:4a740b28-33ae-4e0f-b746-6e5f6cd5eeec nodeName:}" failed. No retries permitted until 2026-07-05 13:38:47.570715898 +0000 UTC m=+3.750558330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret") pod "global-pull-secret-syncer-9wxg4" (UID: "4a740b28-33ae-4e0f-b746-6e5f6cd5eeec") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070770 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-host\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070774 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-cni-bin\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.074609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070807 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-conf-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070832 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-var-lib-openvswitch\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070856 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c57b80fc-d60a-4a29-9101-c71955868288-multus-daemon-config\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070956 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9759e393-3424-4efd-afac-56eedc206f49-host-cni-bin\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.070993 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57b80fc-d60a-4a29-9101-c71955868288-multus-conf-dir\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.071263 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-serviceca\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.073909 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9759e393-3424-4efd-afac-56eedc206f49-ovn-node-metrics-cert\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.075559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.075143 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:38:47.077869 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.077765 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx9pz\" (UniqueName: \"kubernetes.io/projected/43d2952f-50ec-4ce3-9239-0d5d69d5ded0-kube-api-access-tx9pz\") pod \"node-ca-6lmkh\" (UID: \"43d2952f-50ec-4ce3-9239-0d5d69d5ded0\") " pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:47.078147 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.078123 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd756\" (UniqueName: \"kubernetes.io/projected/165d05a1-21b6-4a1f-9069-da352a4aa357-kube-api-access-sd756\") pod \"iptables-alerter-nx8w8\" (UID: \"165d05a1-21b6-4a1f-9069-da352a4aa357\") " pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:47.078859 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.078588 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28g4c\" (UniqueName: \"kubernetes.io/projected/c57b80fc-d60a-4a29-9101-c71955868288-kube-api-access-28g4c\") pod \"multus-srgqx\" (UID: \"c57b80fc-d60a-4a29-9101-c71955868288\") " pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.080180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.080154 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clzl9\" (UniqueName: \"kubernetes.io/projected/c424c3fa-89b7-43e4-80e9-a09466ee2d89-kube-api-access-clzl9\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:47.080180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.080170 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5znfx\" (UniqueName: \"kubernetes.io/projected/9759e393-3424-4efd-afac-56eedc206f49-kube-api-access-5znfx\") pod \"ovnkube-node-dq5h4\" (UID: \"9759e393-3424-4efd-afac-56eedc206f49\") " pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.082655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.082632 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw82v\" (UniqueName: \"kubernetes.io/projected/6a918c1c-3a68-4e16-aa80-ea202e86d206-kube-api-access-cw82v\") pod \"aws-ebs-csi-driver-node-z89v4\" (UID: \"6a918c1c-3a68-4e16-aa80-ea202e86d206\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.095049 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.095023 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zbk46" Jul 05 13:38:47.103300 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.103277 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zbk46" Jul 05 13:38:47.171699 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-var-lib-kubelet\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-lib-modules\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v56n\" (UniqueName: \"kubernetes.io/projected/5730bfca-1a7a-449b-8205-5d938dd443ef-kube-api-access-9v56n\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1f48763-233d-4180-9ae8-cd2e5934ef3b-hosts-file\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysctl-conf\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171841 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-os-release\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-tuned\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171895 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a36113f7-8e41-48ce-9a66-156e196e4c60-konnectivity-ca\") pod \"konnectivity-agent-nmbbx\" (UID: \"a36113f7-8e41-48ce-9a66-156e196e4c60\") " pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171930 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysctl-d\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-systemd\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.171980 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-system-cni-dir\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxfmb\" (UniqueName: \"kubernetes.io/projected/b533b9df-1afd-48fe-8b93-dff258281d4e-kube-api-access-vxfmb\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172035 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-cnibin\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172087 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-kubernetes\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172111 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b533b9df-1afd-48fe-8b93-dff258281d4e-tmp\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a36113f7-8e41-48ce-9a66-156e196e4c60-agent-certs\") pod \"konnectivity-agent-nmbbx\" (UID: \"a36113f7-8e41-48ce-9a66-156e196e4c60\") " pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:47.175124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-modprobe-d\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-sys\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-host\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172324 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172351 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f48763-233d-4180-9ae8-cd2e5934ef3b-tmp-dir\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-run\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172423 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rzxt\" (UniqueName: \"kubernetes.io/projected/d1f48763-233d-4180-9ae8-cd2e5934ef3b-kube-api-access-4rzxt\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysconfig\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172556 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysconfig\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172609 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-cnibin\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.172663 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-kubernetes\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173008 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-host\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173029 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173138 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-var-lib-kubelet\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.176208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173163 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-modprobe-d\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173561 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173632 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-run\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173685 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173718 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1f48763-233d-4180-9ae8-cd2e5934ef3b-tmp-dir\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.173893 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-sys\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174297 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5730bfca-1a7a-449b-8205-5d938dd443ef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174373 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-systemd\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174473 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-system-cni-dir\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174641 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysctl-d\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174747 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-lib-modules\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174763 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1f48763-233d-4180-9ae8-cd2e5934ef3b-hosts-file\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174875 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-sysctl-conf\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.174942 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5730bfca-1a7a-449b-8205-5d938dd443ef-os-release\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.176651 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a36113f7-8e41-48ce-9a66-156e196e4c60-konnectivity-ca\") pod \"konnectivity-agent-nmbbx\" (UID: \"a36113f7-8e41-48ce-9a66-156e196e4c60\") " pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:47.177305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.176812 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b533b9df-1afd-48fe-8b93-dff258281d4e-tmp\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.178830 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.178803 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b533b9df-1afd-48fe-8b93-dff258281d4e-etc-tuned\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.181568 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.180352 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:38:47.181568 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.181287 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:38:47.181568 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.181302 2560 projected.go:196] Error preparing data for projected volume kube-api-access-f78r9 for pod openshift-network-diagnostics/network-check-target-m4t56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:47.181568 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.181376 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9 podName:294d25c3-b89a-4d6a-9475-13ef3a4d8aa5 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:47.681356185 +0000 UTC m=+3.861198631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f78r9" (UniqueName: "kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9") pod "network-check-target-m4t56" (UID: "294d25c3-b89a-4d6a-9475-13ef3a4d8aa5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:47.181848 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.181762 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a36113f7-8e41-48ce-9a66-156e196e4c60-agent-certs\") pod \"konnectivity-agent-nmbbx\" (UID: \"a36113f7-8e41-48ce-9a66-156e196e4c60\") " pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:47.185889 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.185750 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxfmb\" (UniqueName: \"kubernetes.io/projected/b533b9df-1afd-48fe-8b93-dff258281d4e-kube-api-access-vxfmb\") pod \"tuned-r2fpj\" (UID: \"b533b9df-1afd-48fe-8b93-dff258281d4e\") " pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.186195 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.186173 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v56n\" (UniqueName: \"kubernetes.io/projected/5730bfca-1a7a-449b-8205-5d938dd443ef-kube-api-access-9v56n\") pod \"multus-additional-cni-plugins-r4ct4\" (UID: \"5730bfca-1a7a-449b-8205-5d938dd443ef\") " pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.186293 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.186268 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rzxt\" (UniqueName: \"kubernetes.io/projected/d1f48763-233d-4180-9ae8-cd2e5934ef3b-kube-api-access-4rzxt\") pod \"node-resolver-r7c9v\" (UID: \"d1f48763-233d-4180-9ae8-cd2e5934ef3b\") " pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.204324 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.204303 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:38:47.255371 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.255127 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6lmkh" Jul 05 13:38:47.263195 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.263169 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-srgqx" Jul 05 13:38:47.272108 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.272088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nx8w8" Jul 05 13:38:47.274287 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.274257 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d2952f_50ec_4ce3_9239_0d5d69d5ded0.slice/crio-b14a75945390fc0d1333741d0d093677d6afff03f6e2e12e867c7ba17d805326 WatchSource:0}: Error finding container b14a75945390fc0d1333741d0d093677d6afff03f6e2e12e867c7ba17d805326: Status 404 returned error can't find the container with id b14a75945390fc0d1333741d0d093677d6afff03f6e2e12e867c7ba17d805326 Jul 05 13:38:47.275835 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.275522 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57b80fc_d60a_4a29_9101_c71955868288.slice/crio-1151792c77b45b3209fc944e503dfbf16c71a8f58251c196fb8acf2ee241b1c9 WatchSource:0}: Error finding container 1151792c77b45b3209fc944e503dfbf16c71a8f58251c196fb8acf2ee241b1c9: Status 404 returned error can't find the container with id 1151792c77b45b3209fc944e503dfbf16c71a8f58251c196fb8acf2ee241b1c9 Jul 05 13:38:47.276418 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.276231 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:38:47.283704 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.283681 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" Jul 05 13:38:47.284385 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.284318 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165d05a1_21b6_4a1f_9069_da352a4aa357.slice/crio-8bb883e519def07d264b829fca55ec90e4f6e3cb240eec6e839017b2441d44b5 WatchSource:0}: Error finding container 8bb883e519def07d264b829fca55ec90e4f6e3cb240eec6e839017b2441d44b5: Status 404 returned error can't find the container with id 8bb883e519def07d264b829fca55ec90e4f6e3cb240eec6e839017b2441d44b5 Jul 05 13:38:47.287167 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.287126 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9759e393_3424_4efd_afac_56eedc206f49.slice/crio-ae739e673b9899132697b4a3dcf285ce3470760d38c1baad2458604e04928dfe WatchSource:0}: Error finding container ae739e673b9899132697b4a3dcf285ce3470760d38c1baad2458604e04928dfe: Status 404 returned error can't find the container with id ae739e673b9899132697b4a3dcf285ce3470760d38c1baad2458604e04928dfe Jul 05 13:38:47.291943 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.291769 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r7c9v" Jul 05 13:38:47.295215 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.295144 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a918c1c_3a68_4e16_aa80_ea202e86d206.slice/crio-72d70406525712be012997efc0c4317165e3a809d1a4b70ecf5429c25f9dc3be WatchSource:0}: Error finding container 72d70406525712be012997efc0c4317165e3a809d1a4b70ecf5429c25f9dc3be: Status 404 returned error can't find the container with id 72d70406525712be012997efc0c4317165e3a809d1a4b70ecf5429c25f9dc3be Jul 05 13:38:47.299910 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.299821 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" Jul 05 13:38:47.304546 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.304526 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f48763_233d_4180_9ae8_cd2e5934ef3b.slice/crio-a7a1831a5b2ad3a4a18bdce257960474a036da0f09d8be12a43e8de25b9c13c9 WatchSource:0}: Error finding container a7a1831a5b2ad3a4a18bdce257960474a036da0f09d8be12a43e8de25b9c13c9: Status 404 returned error can't find the container with id a7a1831a5b2ad3a4a18bdce257960474a036da0f09d8be12a43e8de25b9c13c9 Jul 05 13:38:47.305808 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.305790 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:38:47.312775 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.312540 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" Jul 05 13:38:47.312775 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.312746 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb533b9df_1afd_48fe_8b93_dff258281d4e.slice/crio-260d2d963c0d7e3dc1da25d1f72c7e50c12ce644fb4bd1359390d6b76f989c8c WatchSource:0}: Error finding container 260d2d963c0d7e3dc1da25d1f72c7e50c12ce644fb4bd1359390d6b76f989c8c: Status 404 returned error can't find the container with id 260d2d963c0d7e3dc1da25d1f72c7e50c12ce644fb4bd1359390d6b76f989c8c Jul 05 13:38:47.318808 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.318780 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36113f7_8e41_48ce_9a66_156e196e4c60.slice/crio-05d67c6c4770cbec4a865afa8dea068ddb97c1c8f309976e17d559f17c61e181 WatchSource:0}: Error finding container 05d67c6c4770cbec4a865afa8dea068ddb97c1c8f309976e17d559f17c61e181: Status 404 returned error can't find the container with id 05d67c6c4770cbec4a865afa8dea068ddb97c1c8f309976e17d559f17c61e181 Jul 05 13:38:47.328140 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:38:47.328088 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5730bfca_1a7a_449b_8205_5d938dd443ef.slice/crio-afc67c72c9fe76ccd458fa09f466695080f02cf7977035c121c429d801217590 WatchSource:0}: Error finding container afc67c72c9fe76ccd458fa09f466695080f02cf7977035c121c429d801217590: Status 404 returned error can't find the container with id afc67c72c9fe76ccd458fa09f466695080f02cf7977035c121c429d801217590 Jul 05 13:38:47.576495 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.576462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:47.576666 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.576508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:47.576666 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.576620 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:47.576793 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.576675 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:47.576793 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.576712 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret podName:4a740b28-33ae-4e0f-b746-6e5f6cd5eeec nodeName:}" failed. No retries permitted until 2026-07-05 13:38:48.576691728 +0000 UTC m=+4.756534160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret") pod "global-pull-secret-syncer-9wxg4" (UID: "4a740b28-33ae-4e0f-b746-6e5f6cd5eeec") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:47.576793 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.576740 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:48.576721227 +0000 UTC m=+4.756563660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:47.590323 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.590290 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nmbbx" event={"ID":"a36113f7-8e41-48ce-9a66-156e196e4c60","Type":"ContainerStarted","Data":"05d67c6c4770cbec4a865afa8dea068ddb97c1c8f309976e17d559f17c61e181"} Jul 05 13:38:47.591500 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.591468 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" event={"ID":"b533b9df-1afd-48fe-8b93-dff258281d4e","Type":"ContainerStarted","Data":"260d2d963c0d7e3dc1da25d1f72c7e50c12ce644fb4bd1359390d6b76f989c8c"} Jul 05 13:38:47.592730 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.592702 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r7c9v" event={"ID":"d1f48763-233d-4180-9ae8-cd2e5934ef3b","Type":"ContainerStarted","Data":"a7a1831a5b2ad3a4a18bdce257960474a036da0f09d8be12a43e8de25b9c13c9"} Jul 05 13:38:47.594008 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.593984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"ae739e673b9899132697b4a3dcf285ce3470760d38c1baad2458604e04928dfe"} Jul 05 13:38:47.595157 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.595135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6lmkh" event={"ID":"43d2952f-50ec-4ce3-9239-0d5d69d5ded0","Type":"ContainerStarted","Data":"b14a75945390fc0d1333741d0d093677d6afff03f6e2e12e867c7ba17d805326"} Jul 05 13:38:47.596363 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.596339 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerStarted","Data":"afc67c72c9fe76ccd458fa09f466695080f02cf7977035c121c429d801217590"} Jul 05 13:38:47.597676 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.597644 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" event={"ID":"6a918c1c-3a68-4e16-aa80-ea202e86d206","Type":"ContainerStarted","Data":"72d70406525712be012997efc0c4317165e3a809d1a4b70ecf5429c25f9dc3be"} Jul 05 13:38:47.598985 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.598960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nx8w8" event={"ID":"165d05a1-21b6-4a1f-9069-da352a4aa357","Type":"ContainerStarted","Data":"8bb883e519def07d264b829fca55ec90e4f6e3cb240eec6e839017b2441d44b5"} Jul 05 13:38:47.600314 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.600293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-srgqx" event={"ID":"c57b80fc-d60a-4a29-9101-c71955868288","Type":"ContainerStarted","Data":"1151792c77b45b3209fc944e503dfbf16c71a8f58251c196fb8acf2ee241b1c9"} Jul 05 13:38:47.601483 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.601456 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" event={"ID":"f2f4209627eb6491cbb2535de8e653eb","Type":"ContainerStarted","Data":"83d846e24125733a7fc851ae2e294bde5b3e48e4f607c22c1047dfcacb8b721a"} Jul 05 13:38:47.777631 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:47.777595 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:47.777849 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.777785 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:38:47.777849 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.777803 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:38:47.777849 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.777814 2560 projected.go:196] Error preparing data for projected volume kube-api-access-f78r9 for pod openshift-network-diagnostics/network-check-target-m4t56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:47.777985 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:47.777869 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9 podName:294d25c3-b89a-4d6a-9475-13ef3a4d8aa5 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:48.777850525 +0000 UTC m=+4.957692961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78r9" (UniqueName: "kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9") pod "network-check-target-m4t56" (UID: "294d25c3-b89a-4d6a-9475-13ef3a4d8aa5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:48.106273 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.106225 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:33:47 +0000 UTC" deadline="2027-11-28 22:13:51.274748238 +0000 UTC" Jul 05 13:38:48.106273 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.106266 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12272h35m3.168487521s" Jul 05 13:38:48.583409 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.583364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:48.583857 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.583452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:48.583857 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.583524 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:48.583857 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.583570 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:48.583857 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.583602 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:50.58358196 +0000 UTC m=+6.763424408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:48.583857 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.583623 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret podName:4a740b28-33ae-4e0f-b746-6e5f6cd5eeec nodeName:}" failed. No retries permitted until 2026-07-05 13:38:50.583612296 +0000 UTC m=+6.763454736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret") pod "global-pull-secret-syncer-9wxg4" (UID: "4a740b28-33ae-4e0f-b746-6e5f6cd5eeec") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:48.586122 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.586098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:48.586248 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.586222 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:38:48.586330 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.586314 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:48.586445 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.586425 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:38:48.586515 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.586497 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:48.586589 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.586571 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:38:48.785830 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:48.785792 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:48.786010 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.785981 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:38:48.786010 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.786006 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:38:48.786130 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.786020 2560 projected.go:196] Error preparing data for projected volume kube-api-access-f78r9 for pod openshift-network-diagnostics/network-check-target-m4t56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:48.786130 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:48.786084 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9 podName:294d25c3-b89a-4d6a-9475-13ef3a4d8aa5 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:50.786062995 +0000 UTC m=+6.965905424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78r9" (UniqueName: "kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9") pod "network-check-target-m4t56" (UID: "294d25c3-b89a-4d6a-9475-13ef3a4d8aa5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:49.107052 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:49.107005 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:33:47 +0000 UTC" deadline="2028-02-21 00:09:04.306283307 +0000 UTC" Jul 05 13:38:49.107052 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:49.107051 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14290h30m15.199238101s" Jul 05 13:38:49.618026 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:49.617089 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" event={"ID":"f2f4209627eb6491cbb2535de8e653eb","Type":"ContainerStarted","Data":"f11a7dabf06160901e818a3a09a63fffdefbb583e9756c5a7ea0fe211ac43ef8"} Jul 05 13:38:49.633730 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:49.633502 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" podStartSLOduration=2.633484232 podStartE2EDuration="2.633484232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:38:49.633337671 +0000 UTC m=+5.813180118" watchObservedRunningTime="2026-07-05 13:38:49.633484232 +0000 UTC m=+5.813326683" Jul 05 13:38:50.585458 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:50.585428 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:50.585634 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.585543 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:38:50.585968 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:50.585948 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:50.586080 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.586055 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:38:50.586146 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:50.586114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:50.586210 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.586191 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:38:50.602696 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:50.602259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:50.602696 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:50.602307 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:50.602696 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.602498 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:50.602696 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.602552 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret podName:4a740b28-33ae-4e0f-b746-6e5f6cd5eeec nodeName:}" failed. No retries permitted until 2026-07-05 13:38:54.60253428 +0000 UTC m=+10.782376710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret") pod "global-pull-secret-syncer-9wxg4" (UID: "4a740b28-33ae-4e0f-b746-6e5f6cd5eeec") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:50.602696 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.602622 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:50.602696 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.602662 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:54.602648566 +0000 UTC m=+10.782490997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:50.625476 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:50.625450 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" event={"ID":"2ff4bc349d7008d7ce7073a381e0e0de","Type":"ContainerStarted","Data":"6da0b479bae408432d8966e71dac481a695b3cff935a0cd7a70a6438832cf385"} Jul 05 13:38:50.804758 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:50.804723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:50.804945 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.804930 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:38:50.805015 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.804955 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:38:50.805015 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.804970 2560 projected.go:196] Error preparing data for projected volume kube-api-access-f78r9 for pod openshift-network-diagnostics/network-check-target-m4t56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:50.805123 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:50.805041 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9 podName:294d25c3-b89a-4d6a-9475-13ef3a4d8aa5 nodeName:}" failed. No retries permitted until 2026-07-05 13:38:54.805021148 +0000 UTC m=+10.984863590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78r9" (UniqueName: "kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9") pod "network-check-target-m4t56" (UID: "294d25c3-b89a-4d6a-9475-13ef3a4d8aa5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:52.585408 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:52.585367 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:52.585880 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:52.585504 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:38:52.585880 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:52.585585 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:52.585880 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:52.585592 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:52.585880 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:52.585787 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:38:52.586117 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:52.585882 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:38:54.586532 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:54.586021 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:54.586532 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.586148 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:38:54.586532 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:54.586497 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:54.586532 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:54.586535 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:54.587173 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.586676 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:38:54.587173 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.586803 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:38:54.638883 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:54.638787 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:54.638883 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:54.638828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:54.639149 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.638957 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:54.639149 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.639024 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret podName:4a740b28-33ae-4e0f-b746-6e5f6cd5eeec nodeName:}" failed. No retries permitted until 2026-07-05 13:39:02.639004672 +0000 UTC m=+18.818847114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret") pod "global-pull-secret-syncer-9wxg4" (UID: "4a740b28-33ae-4e0f-b746-6e5f6cd5eeec") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:38:54.639149 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.638960 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:54.639149 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.639123 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:39:02.63910337 +0000 UTC m=+18.818945813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:38:54.841007 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:54.840919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:54.841181 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.841108 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:38:54.841181 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.841136 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:38:54.841181 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.841150 2560 projected.go:196] Error preparing data for projected volume kube-api-access-f78r9 for pod openshift-network-diagnostics/network-check-target-m4t56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:54.841345 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:54.841216 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9 podName:294d25c3-b89a-4d6a-9475-13ef3a4d8aa5 nodeName:}" failed. No retries permitted until 2026-07-05 13:39:02.841196395 +0000 UTC m=+19.021038837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78r9" (UniqueName: "kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9") pod "network-check-target-m4t56" (UID: "294d25c3-b89a-4d6a-9475-13ef3a4d8aa5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:38:55.641997 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:55.641954 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ff4bc349d7008d7ce7073a381e0e0de" containerID="6da0b479bae408432d8966e71dac481a695b3cff935a0cd7a70a6438832cf385" exitCode=0 Jul 05 13:38:55.642500 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:55.642023 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" event={"ID":"2ff4bc349d7008d7ce7073a381e0e0de","Type":"ContainerDied","Data":"6da0b479bae408432d8966e71dac481a695b3cff935a0cd7a70a6438832cf385"} Jul 05 13:38:56.585920 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:56.585884 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:56.586109 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:56.586033 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:38:56.586109 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:56.585884 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:56.586219 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:56.586157 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:38:56.586219 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:56.585884 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:56.586319 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:56.586243 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:38:58.585902 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:58.585849 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:38:58.586441 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:58.585850 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:38:58.586441 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:58.585974 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:38:58.586441 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:58.586062 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:38:58.586441 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:38:58.585855 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:38:58.586441 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:38:58.586153 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:00.585386 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:00.585353 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:00.585793 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:00.585353 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:00.585793 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:00.585474 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:00.585793 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:00.585353 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:00.585793 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:00.585571 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:00.585793 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:00.585653 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:02.585506 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:02.585475 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:02.585506 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:02.585503 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:02.585970 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.585596 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:02.585970 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:02.585621 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:02.585970 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.585716 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:02.585970 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.585817 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:02.700023 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:02.699990 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:02.700207 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:02.700094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:02.700207 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.700169 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:39:02.700207 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.700195 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:39:02.700369 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.700243 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret podName:4a740b28-33ae-4e0f-b746-6e5f6cd5eeec nodeName:}" failed. No retries permitted until 2026-07-05 13:39:18.700222103 +0000 UTC m=+34.880064537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret") pod "global-pull-secret-syncer-9wxg4" (UID: "4a740b28-33ae-4e0f-b746-6e5f6cd5eeec") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:39:02.700369 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.700263 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:39:18.700252804 +0000 UTC m=+34.880095232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:39:02.901903 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:02.901825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:02.902028 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.901969 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:39:02.902028 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.901984 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:39:02.902028 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.901992 2560 projected.go:196] Error preparing data for projected volume kube-api-access-f78r9 for pod openshift-network-diagnostics/network-check-target-m4t56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:39:02.902154 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:02.902040 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9 podName:294d25c3-b89a-4d6a-9475-13ef3a4d8aa5 nodeName:}" failed. No retries permitted until 2026-07-05 13:39:18.902023925 +0000 UTC m=+35.081866349 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78r9" (UniqueName: "kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9") pod "network-check-target-m4t56" (UID: "294d25c3-b89a-4d6a-9475-13ef3a4d8aa5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:39:04.585305 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:04.585267 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:04.585772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:04.585391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:04.585772 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:04.585398 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:04.585772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:04.585426 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:04.585772 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:04.585510 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:04.585772 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:04.585581 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:06.585368 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:06.585277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:06.585368 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:06.585331 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:06.585872 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:06.585409 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:06.585872 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:06.585410 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:06.585872 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:06.585502 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:06.585872 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:06.585611 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:07.665246 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.665028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r7c9v" event={"ID":"d1f48763-233d-4180-9ae8-cd2e5934ef3b","Type":"ContainerStarted","Data":"a73d37227f75a78ad24dfe1b6269a3cbf2db74a9cbc8366f2c1697bf3d76cb66"} Jul 05 13:39:07.666682 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.666651 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6lmkh" event={"ID":"43d2952f-50ec-4ce3-9239-0d5d69d5ded0","Type":"ContainerStarted","Data":"b0b8b6f5e2cee94e7aea26a9d13b7f30056c4f88fb957c6f8b59eea6500f8da0"} Jul 05 13:39:07.668271 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.668249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerStarted","Data":"3f0c2da9c59920e4b9c127590bcf2fbf1a837f5cafc00f6ad32f904d9d4dae22"} Jul 05 13:39:07.670400 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.670376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" event={"ID":"6a918c1c-3a68-4e16-aa80-ea202e86d206","Type":"ContainerStarted","Data":"bc48be5e6657c208f374f968cfab483768efd5bac1f6e99c9b5020248eb4e2e9"} Jul 05 13:39:07.672061 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.672035 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-srgqx" event={"ID":"c57b80fc-d60a-4a29-9101-c71955868288","Type":"ContainerStarted","Data":"ddc938ae3c27bc05bf7f28b1bc54c982873c72dce685ea626fddd8b6a0cde104"} Jul 05 13:39:07.674070 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.674049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" event={"ID":"2ff4bc349d7008d7ce7073a381e0e0de","Type":"ContainerStarted","Data":"424af61aba8c702fddc31971f10911dde4d905f730894ffa41db836dad848ec6"} Jul 05 13:39:07.675643 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.675614 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nmbbx" event={"ID":"a36113f7-8e41-48ce-9a66-156e196e4c60","Type":"ContainerStarted","Data":"b19bc8e5466a9648339c037a7e8279e00d336ebedb78ef4dc1ee036de8d17f18"} Jul 05 13:39:07.676960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.676939 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" event={"ID":"b533b9df-1afd-48fe-8b93-dff258281d4e","Type":"ContainerStarted","Data":"213ad299ade92376f4cfb6185a4ee779a5979d6913fbbe4190a4aca13be40cfd"} Jul 05 13:39:07.684899 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.684865 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r7c9v" podStartSLOduration=1.753477584 podStartE2EDuration="21.684853298s" totalImagesPullingTime="19.931375714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:07.684422833 +0000 UTC m=+23.864265290" watchObservedRunningTime="2026-07-05 13:39:07.684853298 +0000 UTC m=+23.864695743" Jul 05 13:39:07.701407 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.701359 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nmbbx" podStartSLOduration=2.006314712 podStartE2EDuration="21.701340076s" totalImagesPullingTime="19.695025364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:07.70043375 +0000 UTC m=+23.880276196" watchObservedRunningTime="2026-07-05 13:39:07.701340076 +0000 UTC m=+23.881182523" Jul 05 13:39:07.716385 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.716349 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" podStartSLOduration=20.716339992 podStartE2EDuration="20.716339992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:07.715873727 +0000 UTC m=+23.895716173" watchObservedRunningTime="2026-07-05 13:39:07.716339992 +0000 UTC m=+23.896182435" Jul 05 13:39:07.731387 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.731347 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6lmkh" podStartSLOduration=1.743304014 podStartE2EDuration="21.731334567s" totalImagesPullingTime="19.988030553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:07.730810555 +0000 UTC m=+23.910653002" watchObservedRunningTime="2026-07-05 13:39:07.731334567 +0000 UTC m=+23.911177003" Jul 05 13:39:07.750778 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.750739 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-srgqx" podStartSLOduration=1.731007162 podStartE2EDuration="21.750727644s" totalImagesPullingTime="20.019720482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:07.750349376 +0000 UTC m=+23.930191822" watchObservedRunningTime="2026-07-05 13:39:07.750727644 +0000 UTC m=+23.930570090" Jul 05 13:39:07.791440 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:07.791401 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r2fpj" podStartSLOduration=1.8245472029999998 podStartE2EDuration="21.79138547s" totalImagesPullingTime="19.966838267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:07.790989345 +0000 UTC m=+23.970831799" watchObservedRunningTime="2026-07-05 13:39:07.79138547 +0000 UTC m=+23.971227916" Jul 05 13:39:08.585530 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.585296 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:08.585637 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.585360 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:08.585699 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:08.585626 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:08.585757 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:08.585717 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:08.585757 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.585363 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:08.585859 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:08.585843 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:08.640492 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.640463 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 13:39:08.682890 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.682869 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:39:08.683655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.683212 2560 generic.go:356] "Generic (PLEG): container finished" podID="9759e393-3424-4efd-afac-56eedc206f49" containerID="8da2b95282e7c21b4482684a74243886b3b7212acb860866c19ec0fcbe4c1337" exitCode=1 Jul 05 13:39:08.683655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.683253 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"f6e5185db2649055be16796fb71bdea60dc95e4e063154836a04e0f09fe75d72"} Jul 05 13:39:08.683655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.683359 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"654f6101a97fa8fb9ee6f7bd69074416073e6ca2a26285a9c1aa1c85376a4dd9"} Jul 05 13:39:08.683655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.683414 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"60068b235098e1c2920604640d02a0659306a0ea172cae90fbc1d2108d675301"} Jul 05 13:39:08.683655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.683463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"af3123e661a520a4d900000aa0aac60938711f8c5b9002b0bf6f6dac9cca2a17"} Jul 05 13:39:08.683655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.683510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerDied","Data":"8da2b95282e7c21b4482684a74243886b3b7212acb860866c19ec0fcbe4c1337"} Jul 05 13:39:08.683655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.683570 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"7eb4f74df8f52af901c43614db54c7b2ea54652307890300d8f12c7215efd75e"} Jul 05 13:39:08.684949 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.684926 2560 generic.go:356] "Generic (PLEG): container finished" podID="5730bfca-1a7a-449b-8205-5d938dd443ef" containerID="3f0c2da9c59920e4b9c127590bcf2fbf1a837f5cafc00f6ad32f904d9d4dae22" exitCode=0 Jul 05 13:39:08.685062 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.684996 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerDied","Data":"3f0c2da9c59920e4b9c127590bcf2fbf1a837f5cafc00f6ad32f904d9d4dae22"} Jul 05 13:39:08.686922 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.686896 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" event={"ID":"6a918c1c-3a68-4e16-aa80-ea202e86d206","Type":"ContainerStarted","Data":"89f33f9098ffe236dd030afabbb54bb76698d21a53ed612ebc8df7867818aaf1"} Jul 05 13:39:08.688463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.688436 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nx8w8" event={"ID":"165d05a1-21b6-4a1f-9069-da352a4aa357","Type":"ContainerStarted","Data":"5fe9cd2e310b7ebf1ce8858b5c0aec00071961f9316ae5ffb24751234010ddd1"} Jul 05 13:39:08.703839 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.703792 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nx8w8" podStartSLOduration=2.973060797 podStartE2EDuration="22.703774355s" totalImagesPullingTime="19.730713558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:08.703199378 +0000 UTC m=+24.883041825" watchObservedRunningTime="2026-07-05 13:39:08.703774355 +0000 UTC m=+24.883616802" Jul 05 13:39:08.845394 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.845356 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:08.931628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:08.931600 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:09.032754 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:09.032724 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:09.549379 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:09.549260 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T13:39:08.640482143Z","UUID":"3e8b7ba5-e2da-4fc3-95d2-d1194c6cc7ae","Handler":null,"Name":"","Endpoint":""} Jul 05 13:39:09.550830 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:09.550795 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 13:39:09.550957 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:09.550854 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 13:39:10.585218 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:10.585021 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:10.585620 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:10.585060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:10.585620 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:10.585328 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:10.585620 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:10.585092 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:10.585620 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:10.585388 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:10.585620 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:10.585453 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:10.695662 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:10.695583 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" event={"ID":"6a918c1c-3a68-4e16-aa80-ea202e86d206","Type":"ContainerStarted","Data":"3edc981c591eeec747a17c676c3a313e949562ae4a49c0a4ca5b0222ed4635c7"} Jul 05 13:39:10.698961 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:10.698942 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:39:10.699339 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:10.699303 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"2fdc518eaf14101a41bd2fb7b253b2dcf4017158cf595978d8ad67f1db051b64"} Jul 05 13:39:10.715784 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:10.715738 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z89v4" podStartSLOduration=2.725207386 podStartE2EDuration="24.71572226s" totalImagesPullingTime="21.990514874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:10.715226158 +0000 UTC m=+26.895068618" watchObservedRunningTime="2026-07-05 13:39:10.71572226 +0000 UTC m=+26.895564707" Jul 05 13:39:11.329126 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:11.329089 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:12.584229 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.584206 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:39:12.584907 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.584723 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:39:12.584981 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.584937 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:12.585032 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.585007 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:12.585032 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.585023 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:12.585111 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:12.585016 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:12.585111 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:12.585078 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:12.585193 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:12.585153 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:12.708185 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.707817 2560 generic.go:356] "Generic (PLEG): container finished" podID="5730bfca-1a7a-449b-8205-5d938dd443ef" containerID="1646e79b1c139346810cb5da7a12cd0c947d1c877a791e769558c6457042326e" exitCode=0 Jul 05 13:39:12.708185 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.707898 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerDied","Data":"1646e79b1c139346810cb5da7a12cd0c947d1c877a791e769558c6457042326e"} Jul 05 13:39:12.712646 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.712581 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:39:12.713265 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.712974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"f41ea1e7825a03347d39c6826dac34d4c9a84f528124de94c36cc24c5276a4f8"} Jul 05 13:39:12.713265 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.713202 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:39:12.713265 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.713265 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:39:12.713674 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.713441 2560 scope.go:117] "RemoveContainer" containerID="8da2b95282e7c21b4482684a74243886b3b7212acb860866c19ec0fcbe4c1337" Jul 05 13:39:12.714333 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.713899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nmbbx" Jul 05 13:39:12.730221 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.730204 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:39:12.817136 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:12.817109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:13.721238 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:13.721213 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:39:13.721798 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:13.721683 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" event={"ID":"9759e393-3424-4efd-afac-56eedc206f49","Type":"ContainerStarted","Data":"90e76e0066cf5f231c9765b0fb8bfcf61d5aefe95d0a4d0ab2305d51a42369be"} Jul 05 13:39:13.721863 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:13.721795 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:39:13.721930 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:13.721916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:39:13.736688 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:13.736669 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:39:13.750240 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:13.750201 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" podStartSLOduration=7.366855475 podStartE2EDuration="27.750190358s" totalImagesPullingTime="20.383334883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:13.74859913 +0000 UTC m=+29.928441577" watchObservedRunningTime="2026-07-05 13:39:13.750190358 +0000 UTC m=+29.930032829" Jul 05 13:39:14.105187 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.105160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:14.328147 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.328108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:14.427603 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.427572 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:14.585067 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.585015 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:14.585177 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.585015 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:14.585177 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:14.585122 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:14.585270 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.585020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:14.585270 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:14.585176 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:14.585270 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:14.585263 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:14.698343 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.698304 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9wxg4"] Jul 05 13:39:14.701628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.701445 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m4t56"] Jul 05 13:39:14.702246 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.702222 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n66xq"] Jul 05 13:39:14.726698 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.726672 2560 generic.go:356] "Generic (PLEG): container finished" podID="5730bfca-1a7a-449b-8205-5d938dd443ef" containerID="f9a5de4079e499fa4ed96c9365cf138862a51155f110b8d4839bac9ba89a9682" exitCode=0 Jul 05 13:39:14.727058 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.726730 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerDied","Data":"f9a5de4079e499fa4ed96c9365cf138862a51155f110b8d4839bac9ba89a9682"} Jul 05 13:39:14.727058 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.726887 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:14.727238 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:14.727214 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:14.727238 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.727231 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:14.727422 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:14.727331 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:14.727422 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:14.727351 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:14.727511 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:14.727455 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:16.585715 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:16.585685 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:16.586342 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:16.585791 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:16.586342 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:16.585799 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:16.586342 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:16.585819 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:16.586342 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:16.585887 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:16.586342 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:16.585950 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:16.732700 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:16.732671 2560 generic.go:356] "Generic (PLEG): container finished" podID="5730bfca-1a7a-449b-8205-5d938dd443ef" containerID="2564b4e9ba4f0c9985b92f3f4dc0b568e8d18c72c1c7511147def4baf7db821e" exitCode=0 Jul 05 13:39:16.732807 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:16.732722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerDied","Data":"2564b4e9ba4f0c9985b92f3f4dc0b568e8d18c72c1c7511147def4baf7db821e"} Jul 05 13:39:18.585850 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:18.585819 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:18.586560 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:18.585866 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:18.586560 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.585932 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:18.586560 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.585992 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:18.586560 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:18.586031 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:18.586560 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.586129 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:18.737844 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:18.737814 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:18.738000 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:18.737848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:18.738000 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.737970 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:39:18.738106 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.738024 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret podName:4a740b28-33ae-4e0f-b746-6e5f6cd5eeec nodeName:}" failed. No retries permitted until 2026-07-05 13:39:50.738006129 +0000 UTC m=+66.917848568 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret") pod "global-pull-secret-syncer-9wxg4" (UID: "4a740b28-33ae-4e0f-b746-6e5f6cd5eeec") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:39:18.738106 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.737971 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:39:18.738189 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.738108 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:39:50.738089321 +0000 UTC m=+66.917931762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:39:18.939507 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:18.939423 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:18.939674 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.939563 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:39:18.939674 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.939584 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:39:18.939674 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.939596 2560 projected.go:196] Error preparing data for projected volume kube-api-access-f78r9 for pod openshift-network-diagnostics/network-check-target-m4t56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:39:18.939674 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:18.939651 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9 podName:294d25c3-b89a-4d6a-9475-13ef3a4d8aa5 nodeName:}" failed. No retries permitted until 2026-07-05 13:39:50.93963343 +0000 UTC m=+67.119475869 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78r9" (UniqueName: "kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9") pod "network-check-target-m4t56" (UID: "294d25c3-b89a-4d6a-9475-13ef3a4d8aa5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:39:19.279390 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:19.279356 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:19.334532 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:19.334497 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:20.585944 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:20.585914 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:20.586443 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:20.585916 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:20.586443 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:20.586028 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:20.586443 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:20.586019 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4t56" podUID="294d25c3-b89a-4d6a-9475-13ef3a4d8aa5" Jul 05 13:39:20.586443 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:20.586141 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9wxg4" podUID="4a740b28-33ae-4e0f-b746-6e5f6cd5eeec" Jul 05 13:39:20.586443 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:20.586223 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:39:21.064329 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.064289 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeReady" Jul 05 13:39:21.064445 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.064435 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 13:39:21.114003 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.113977 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-77lql"] Jul 05 13:39:21.139439 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.139415 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hgq7h"] Jul 05 13:39:21.139558 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.139534 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.142620 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.142464 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sw56z\"" Jul 05 13:39:21.142718 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.142488 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 13:39:21.142718 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.142488 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 13:39:21.154952 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.154925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-tmp-dir\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.155045 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.154976 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.155102 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.155075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hh89\" (UniqueName: \"kubernetes.io/projected/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-kube-api-access-8hh89\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.155149 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.155121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-config-volume\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.159490 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.159469 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hgq7h"] Jul 05 13:39:21.159575 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.159495 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-77lql"] Jul 05 13:39:21.159575 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.159568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.162230 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.162212 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 13:39:21.162340 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.162216 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mls4n\"" Jul 05 13:39:21.162340 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.162215 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 13:39:21.162453 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.162389 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 13:39:21.255822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.255800 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-tmp-dir\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.255914 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.255830 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.255914 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.255852 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a5d4105-f02a-4be3-aafa-e7edb24c1163-cert\") pod \"ingress-canary-hgq7h\" (UID: \"8a5d4105-f02a-4be3-aafa-e7edb24c1163\") " pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.255989 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:21.255972 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:39:21.256061 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:21.256041 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:39:21.756017257 +0000 UTC m=+37.935859681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:39:21.256061 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.255979 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hh89\" (UniqueName: \"kubernetes.io/projected/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-kube-api-access-8hh89\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.256168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.256093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-config-volume\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.256168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.256122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s6xz\" (UniqueName: \"kubernetes.io/projected/8a5d4105-f02a-4be3-aafa-e7edb24c1163-kube-api-access-6s6xz\") pod \"ingress-canary-hgq7h\" (UID: \"8a5d4105-f02a-4be3-aafa-e7edb24c1163\") " pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.256168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.256150 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-tmp-dir\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.256731 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.256708 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-config-volume\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.265650 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.265629 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hh89\" (UniqueName: \"kubernetes.io/projected/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-kube-api-access-8hh89\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.356736 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.356708 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a5d4105-f02a-4be3-aafa-e7edb24c1163-cert\") pod \"ingress-canary-hgq7h\" (UID: \"8a5d4105-f02a-4be3-aafa-e7edb24c1163\") " pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.356850 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.356763 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s6xz\" (UniqueName: \"kubernetes.io/projected/8a5d4105-f02a-4be3-aafa-e7edb24c1163-kube-api-access-6s6xz\") pod \"ingress-canary-hgq7h\" (UID: \"8a5d4105-f02a-4be3-aafa-e7edb24c1163\") " pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.358951 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.358925 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a5d4105-f02a-4be3-aafa-e7edb24c1163-cert\") pod \"ingress-canary-hgq7h\" (UID: \"8a5d4105-f02a-4be3-aafa-e7edb24c1163\") " pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.365073 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.365056 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s6xz\" (UniqueName: \"kubernetes.io/projected/8a5d4105-f02a-4be3-aafa-e7edb24c1163-kube-api-access-6s6xz\") pod \"ingress-canary-hgq7h\" (UID: \"8a5d4105-f02a-4be3-aafa-e7edb24c1163\") " pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.467515 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.467454 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hgq7h" Jul 05 13:39:21.597936 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.597907 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hgq7h"] Jul 05 13:39:21.602205 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:39:21.602183 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5d4105_f02a_4be3_aafa_e7edb24c1163.slice/crio-28a52011ee77cab5c6b80b59b3679957e72dfd7210787d118e9d81c993d4e6f7 WatchSource:0}: Error finding container 28a52011ee77cab5c6b80b59b3679957e72dfd7210787d118e9d81c993d4e6f7: Status 404 returned error can't find the container with id 28a52011ee77cab5c6b80b59b3679957e72dfd7210787d118e9d81c993d4e6f7 Jul 05 13:39:21.745886 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.745825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hgq7h" event={"ID":"8a5d4105-f02a-4be3-aafa-e7edb24c1163","Type":"ContainerStarted","Data":"28a52011ee77cab5c6b80b59b3679957e72dfd7210787d118e9d81c993d4e6f7"} Jul 05 13:39:21.748302 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.748278 2560 generic.go:356] "Generic (PLEG): container finished" podID="5730bfca-1a7a-449b-8205-5d938dd443ef" containerID="3334fe296fdd60edca0216d52d6d791dc81c58d7c88dc4aef3af7ddd7916ca3f" exitCode=0 Jul 05 13:39:21.748407 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.748341 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerDied","Data":"3334fe296fdd60edca0216d52d6d791dc81c58d7c88dc4aef3af7ddd7916ca3f"} Jul 05 13:39:21.760746 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:21.760722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:21.760854 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:21.760810 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:39:21.760911 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:21.760870 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:39:22.760849722 +0000 UTC m=+38.940692160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:39:22.465097 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.465060 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:22.522390 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.522334 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:22.585133 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.585101 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:22.585133 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.585115 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:22.585457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.585422 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:22.589535 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.589294 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:39:22.589535 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.589305 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xhx9m\"" Jul 05 13:39:22.589535 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.589337 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:39:22.589535 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.589474 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:39:22.589758 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.589603 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fvggw\"" Jul 05 13:39:22.589758 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.589641 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:39:22.754880 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.754794 2560 generic.go:356] "Generic (PLEG): container finished" podID="5730bfca-1a7a-449b-8205-5d938dd443ef" containerID="3aefdf3b277335056f5adff00d4c92135dfca8779402828bf4ea1901bc3c0172" exitCode=0 Jul 05 13:39:22.754880 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.754828 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerDied","Data":"3aefdf3b277335056f5adff00d4c92135dfca8779402828bf4ea1901bc3c0172"} Jul 05 13:39:22.767323 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:22.767133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:22.767987 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:22.767967 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:39:22.768070 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:22.768026 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:39:24.768005953 +0000 UTC m=+40.947848384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:39:23.760066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:23.760041 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" event={"ID":"5730bfca-1a7a-449b-8205-5d938dd443ef","Type":"ContainerStarted","Data":"37048e71cb57e8fded8ca43ce8bb66589762f56b575660c2c98e0467814b946d"} Jul 05 13:39:23.787486 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:23.787442 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-r4ct4" podStartSLOduration=6.890289024 podStartE2EDuration="37.787427984s" totalImagesPullingTime="30.89713896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:23.785640377 +0000 UTC m=+39.965482824" watchObservedRunningTime="2026-07-05 13:39:23.787427984 +0000 UTC m=+39.967270430" Jul 05 13:39:23.908710 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:23.908674 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:23.971553 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:23.971477 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:24.449707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:24.449671 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:24.547786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:24.547696 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:24.764049 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:24.764016 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hgq7h" event={"ID":"8a5d4105-f02a-4be3-aafa-e7edb24c1163","Type":"ContainerStarted","Data":"3bbd7c729bd3dd96ec2337b3c654b3e9bcd8135ed71396e31f6d5859b623b6e7"} Jul 05 13:39:24.780352 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:24.780291 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hgq7h" podStartSLOduration=1.6780968189999999 podStartE2EDuration="3.780275007s" totalImagesPullingTime="2.102178188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:39:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:24.77979716 +0000 UTC m=+40.959639608" watchObservedRunningTime="2026-07-05 13:39:24.780275007 +0000 UTC m=+40.960117453" Jul 05 13:39:24.784253 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:24.784220 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:24.784496 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:24.784476 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:39:24.784556 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:24.784547 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:39:28.784529329 +0000 UTC m=+44.964371757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:39:28.809815 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:28.809780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:28.810210 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:28.809898 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:39:28.810210 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:28.809960 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:39:36.809946629 +0000 UTC m=+52.989789053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:39:34.287994 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:34.287955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:34.343616 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:34.343585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:36.861712 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:36.861670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:36.862106 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:36.861805 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:39:36.862106 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:36.861865 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:39:52.861850025 +0000 UTC m=+69.041692449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:39:44.436488 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:44.436445 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:44.536448 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:44.536419 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:45.738571 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:45.738541 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dq5h4" Jul 05 13:39:45.854200 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:45.854166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:45.911646 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:45.911613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:45.966551 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:45.966515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:48.494073 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:48.494033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:48.551070 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:48.551038 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:39:50.756068 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.756023 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:39:50.756068 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.756072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:50.759181 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.759163 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:39:50.759229 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.759203 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:39:50.766540 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:50.766522 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 13:39:50.766601 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:50.766589 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:40:54.766570473 +0000 UTC m=+130.946412896 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : secret "metrics-daemon-secret" not found Jul 05 13:39:50.769757 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.769739 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a740b28-33ae-4e0f-b746-6e5f6cd5eeec-original-pull-secret\") pod \"global-pull-secret-syncer-9wxg4\" (UID: \"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec\") " pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:50.802647 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.802610 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9wxg4" Jul 05 13:39:50.923061 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.923026 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9wxg4"] Jul 05 13:39:50.926621 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:39:50.926586 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a740b28_33ae_4e0f_b746_6e5f6cd5eeec.slice/crio-fa8e331367183969fa7ea856858b618f2525b912e238451a2ec48d8b04132c5b WatchSource:0}: Error finding container fa8e331367183969fa7ea856858b618f2525b912e238451a2ec48d8b04132c5b: Status 404 returned error can't find the container with id fa8e331367183969fa7ea856858b618f2525b912e238451a2ec48d8b04132c5b Jul 05 13:39:50.957303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.957279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:50.960553 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.960533 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:39:50.970458 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.970435 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:39:50.981156 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:50.981126 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f78r9\" (UniqueName: \"kubernetes.io/projected/294d25c3-b89a-4d6a-9475-13ef3a4d8aa5-kube-api-access-f78r9\") pod \"network-check-target-m4t56\" (UID: \"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5\") " pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:51.099216 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:51.099190 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xhx9m\"" Jul 05 13:39:51.106669 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:51.106650 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:51.225942 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:51.225910 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m4t56"] Jul 05 13:39:51.228638 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:39:51.228614 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294d25c3_b89a_4d6a_9475_13ef3a4d8aa5.slice/crio-7730aaa80ac3f6e6248f51aa3519c61c2307eed38950a2774ec42fc2b7554874 WatchSource:0}: Error finding container 7730aaa80ac3f6e6248f51aa3519c61c2307eed38950a2774ec42fc2b7554874: Status 404 returned error can't find the container with id 7730aaa80ac3f6e6248f51aa3519c61c2307eed38950a2774ec42fc2b7554874 Jul 05 13:39:51.826722 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:51.826677 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m4t56" event={"ID":"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5","Type":"ContainerStarted","Data":"7730aaa80ac3f6e6248f51aa3519c61c2307eed38950a2774ec42fc2b7554874"} Jul 05 13:39:51.828079 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:51.828049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9wxg4" event={"ID":"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec","Type":"ContainerStarted","Data":"fa8e331367183969fa7ea856858b618f2525b912e238451a2ec48d8b04132c5b"} Jul 05 13:39:52.872190 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:52.872136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:39:52.872678 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:52.872349 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:39:52.872678 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:39:52.872432 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:40:24.872408212 +0000 UTC m=+101.052250640 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:39:55.521643 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.521603 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs"] Jul 05 13:39:55.536937 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.536909 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd"] Jul 05 13:39:55.537069 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.537048 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.539832 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.539794 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 05 13:39:55.539968 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.539815 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 05 13:39:55.539968 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.539848 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 05 13:39:55.539968 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.539872 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-z8pvs\"" Jul 05 13:39:55.540189 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.540174 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 05 13:39:55.551057 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.551040 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs"] Jul 05 13:39:55.551149 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.551062 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd"] Jul 05 13:39:55.551149 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.551125 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.553764 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.553745 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 05 13:39:55.553830 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.553800 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 05 13:39:55.553872 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.553847 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 05 13:39:55.553995 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.553983 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 05 13:39:55.591947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.591924 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/651da58e-2c5b-4149-9e6f-dc781271e7b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b54848dcd-s8wvs\" (UID: \"651da58e-2c5b-4149-9e6f-dc781271e7b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.592028 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.591956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.592028 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.592011 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f09c9d-737a-4183-8f45-e4147d0b832b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.592100 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.592042 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-ca\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.592100 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.592063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-hub\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.592100 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.592083 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt8b5\" (UniqueName: \"kubernetes.io/projected/89f09c9d-737a-4183-8f45-e4147d0b832b-kube-api-access-tt8b5\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.592188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.592107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks2sw\" (UniqueName: \"kubernetes.io/projected/651da58e-2c5b-4149-9e6f-dc781271e7b6-kube-api-access-ks2sw\") pod \"managed-serviceaccount-addon-agent-5b54848dcd-s8wvs\" (UID: \"651da58e-2c5b-4149-9e6f-dc781271e7b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.592188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.592129 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.693323 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt8b5\" (UniqueName: \"kubernetes.io/projected/89f09c9d-737a-4183-8f45-e4147d0b832b-kube-api-access-tt8b5\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.693410 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks2sw\" (UniqueName: \"kubernetes.io/projected/651da58e-2c5b-4149-9e6f-dc781271e7b6-kube-api-access-ks2sw\") pod \"managed-serviceaccount-addon-agent-5b54848dcd-s8wvs\" (UID: \"651da58e-2c5b-4149-9e6f-dc781271e7b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.693410 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.693410 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693394 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/651da58e-2c5b-4149-9e6f-dc781271e7b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b54848dcd-s8wvs\" (UID: \"651da58e-2c5b-4149-9e6f-dc781271e7b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.693518 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.693518 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f09c9d-737a-4183-8f45-e4147d0b832b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.693634 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693613 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-ca\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.693684 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.693651 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-hub\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.694145 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.694125 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f09c9d-737a-4183-8f45-e4147d0b832b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.695952 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.695924 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.696342 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.696305 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-ca\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.696412 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.696379 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.696491 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.696470 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/89f09c9d-737a-4183-8f45-e4147d0b832b-hub\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.696551 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.696536 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/651da58e-2c5b-4149-9e6f-dc781271e7b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b54848dcd-s8wvs\" (UID: \"651da58e-2c5b-4149-9e6f-dc781271e7b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.701846 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.701827 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks2sw\" (UniqueName: \"kubernetes.io/projected/651da58e-2c5b-4149-9e6f-dc781271e7b6-kube-api-access-ks2sw\") pod \"managed-serviceaccount-addon-agent-5b54848dcd-s8wvs\" (UID: \"651da58e-2c5b-4149-9e6f-dc781271e7b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.701951 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.701914 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt8b5\" (UniqueName: \"kubernetes.io/projected/89f09c9d-737a-4183-8f45-e4147d0b832b-kube-api-access-tt8b5\") pod \"cluster-proxy-proxy-agent-8459f56944-5nggd\" (UID: \"89f09c9d-737a-4183-8f45-e4147d0b832b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:55.862161 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.862142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" Jul 05 13:39:55.868751 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:55.868730 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:39:56.008831 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.008767 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs"] Jul 05 13:39:56.012335 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:39:56.012292 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod651da58e_2c5b_4149_9e6f_dc781271e7b6.slice/crio-af32e92d87f58ea5192c15e7f3884566b820ef58956f00d0f36dcfdcdb728ee9 WatchSource:0}: Error finding container af32e92d87f58ea5192c15e7f3884566b820ef58956f00d0f36dcfdcdb728ee9: Status 404 returned error can't find the container with id af32e92d87f58ea5192c15e7f3884566b820ef58956f00d0f36dcfdcdb728ee9 Jul 05 13:39:56.055599 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.055529 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd"] Jul 05 13:39:56.057761 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:39:56.057740 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89f09c9d_737a_4183_8f45_e4147d0b832b.slice/crio-63cdfb621f1a132d6955cf96df7192f113f8e188c5780ffda9fc39532f58b778 WatchSource:0}: Error finding container 63cdfb621f1a132d6955cf96df7192f113f8e188c5780ffda9fc39532f58b778: Status 404 returned error can't find the container with id 63cdfb621f1a132d6955cf96df7192f113f8e188c5780ffda9fc39532f58b778 Jul 05 13:39:56.842884 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.842843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m4t56" event={"ID":"294d25c3-b89a-4d6a-9475-13ef3a4d8aa5","Type":"ContainerStarted","Data":"dd169a220d17e839d92af3b65b921046ed769c90f7896cae71fd6c1b6aee8cdd"} Jul 05 13:39:56.843534 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.842963 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:39:56.844630 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.844603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" event={"ID":"89f09c9d-737a-4183-8f45-e4147d0b832b","Type":"ContainerStarted","Data":"63cdfb621f1a132d6955cf96df7192f113f8e188c5780ffda9fc39532f58b778"} Jul 05 13:39:56.846204 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.846172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" event={"ID":"651da58e-2c5b-4149-9e6f-dc781271e7b6","Type":"ContainerStarted","Data":"af32e92d87f58ea5192c15e7f3884566b820ef58956f00d0f36dcfdcdb728ee9"} Jul 05 13:39:56.848329 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.848074 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9wxg4" event={"ID":"4a740b28-33ae-4e0f-b746-6e5f6cd5eeec","Type":"ContainerStarted","Data":"bd121003515c0ca8e5fabfea34f86036aaf082004fc58d55938f9ef73dcd846c"} Jul 05 13:39:56.873583 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.873535 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m4t56" podStartSLOduration=66.349474047 podStartE2EDuration="1m10.873518274s" totalImagesPullingTime="4.524044227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:56.859198846 +0000 UTC m=+73.039041292" watchObservedRunningTime="2026-07-05 13:39:56.873518274 +0000 UTC m=+73.053360723" Jul 05 13:39:56.873966 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:56.873922 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9wxg4" podStartSLOduration=66.037194174 podStartE2EDuration="1m10.873911198s" totalImagesPullingTime="4.836717024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:56.873185386 +0000 UTC m=+73.053027835" watchObservedRunningTime="2026-07-05 13:39:56.873911198 +0000 UTC m=+73.053753643" Jul 05 13:39:58.854094 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:58.854062 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" event={"ID":"651da58e-2c5b-4149-9e6f-dc781271e7b6","Type":"ContainerStarted","Data":"316c258ba904a06017c5e998bd334620565974721aa5d84a6a913d822ef3ba4a"} Jul 05 13:39:58.871404 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:39:58.871366 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" podStartSLOduration=1.471847025 podStartE2EDuration="3.871351778s" totalImagesPullingTime="2.399504753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:39:58.87065982 +0000 UTC m=+75.050502272" watchObservedRunningTime="2026-07-05 13:39:58.871351778 +0000 UTC m=+75.051194224" Jul 05 13:40:02.866409 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:02.866374 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" event={"ID":"89f09c9d-737a-4183-8f45-e4147d0b832b","Type":"ContainerStarted","Data":"6ef340bf7e218ffb3a500f989e0fbb00d2eb87edff0eab1da6b1ecc662619678"} Jul 05 13:40:02.866801 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:02.866422 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" event={"ID":"89f09c9d-737a-4183-8f45-e4147d0b832b","Type":"ContainerStarted","Data":"fef54500cc8487e20205c8aa946861708c37ba043a5f069e443835b2b96bd680"} Jul 05 13:40:02.866801 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:02.866440 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" event={"ID":"89f09c9d-737a-4183-8f45-e4147d0b832b","Type":"ContainerStarted","Data":"8e5e3de7b76b760ed607275c72b73f3381937c92e86b2097c003ea13beacc0c7"} Jul 05 13:40:02.887190 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:02.887141 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" podStartSLOduration=2.10898402 podStartE2EDuration="7.887126139s" totalImagesPullingTime="5.778142119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:40:02.885760543 +0000 UTC m=+79.065602991" watchObservedRunningTime="2026-07-05 13:40:02.887126139 +0000 UTC m=+79.066968584" Jul 05 13:40:20.485788 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:20.485675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:40:20.544864 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:20.544826 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:40:24.882577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:24.882543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:40:24.882908 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:40:24.882658 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:40:24.882908 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:40:24.882718 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:41:28.882700074 +0000 UTC m=+165.062542501 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:40:27.852623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:27.852591 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m4t56" Jul 05 13:40:54.784035 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:40:54.783991 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:40:54.784451 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:40:54.784139 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 13:40:54.784451 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:40:54.784204 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:42:56.784186558 +0000 UTC m=+252.964028987 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : secret "metrics-daemon-secret" not found Jul 05 13:41:24.148363 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:41:24.148296 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-77lql" podUID="3908baf2-7deb-42c1-8ccb-71a7a3008b0d" Jul 05 13:41:25.098224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:41:25.098193 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-77lql" Jul 05 13:41:25.607980 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:41:25.607946 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:41:28.899291 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:41:28.899248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:41:28.899691 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:41:28.899384 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:41:28.899691 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:41:28.899438 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls podName:3908baf2-7deb-42c1-8ccb-71a7a3008b0d nodeName:}" failed. No retries permitted until 2026-07-05 13:43:30.899424627 +0000 UTC m=+287.079267051 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls") pod "dns-default-77lql" (UID: "3908baf2-7deb-42c1-8ccb-71a7a3008b0d") : secret "dns-default-metrics-tls" not found Jul 05 13:41:39.586060 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:41:39.585979 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:41:59.197204 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:41:59.197171 2560 generic.go:356] "Generic (PLEG): container finished" podID="651da58e-2c5b-4149-9e6f-dc781271e7b6" containerID="316c258ba904a06017c5e998bd334620565974721aa5d84a6a913d822ef3ba4a" exitCode=255 Jul 05 13:41:59.197610 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:41:59.197233 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" event={"ID":"651da58e-2c5b-4149-9e6f-dc781271e7b6","Type":"ContainerDied","Data":"316c258ba904a06017c5e998bd334620565974721aa5d84a6a913d822ef3ba4a"} Jul 05 13:41:59.197610 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:41:59.197491 2560 scope.go:117] "RemoveContainer" containerID="316c258ba904a06017c5e998bd334620565974721aa5d84a6a913d822ef3ba4a" Jul 05 13:42:00.201910 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:00.201880 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b54848dcd-s8wvs" event={"ID":"651da58e-2c5b-4149-9e6f-dc781271e7b6","Type":"ContainerStarted","Data":"ec24cd826f1db599901f1398a443599882b7fe06d1643b0c62246383fb375a56"} Jul 05 13:42:29.159880 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:29.159852 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:55.823490 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:55.823453 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:55.924636 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:55.924597 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:56.857446 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:56.857415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:42:56.857808 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:42:56.857546 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 13:42:56.857808 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:42:56.857617 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs podName:c424c3fa-89b7-43e4-80e9-a09466ee2d89 nodeName:}" failed. No retries permitted until 2026-07-05 13:44:58.857602198 +0000 UTC m=+375.037444623 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs") pod "network-metrics-daemon-n66xq" (UID: "c424c3fa-89b7-43e4-80e9-a09466ee2d89") : secret "metrics-daemon-secret" not found Jul 05 13:42:57.940457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:57.940425 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:58.000129 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:58.000103 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:59.380913 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.380878 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:59.480945 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.480914 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:59.659629 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.659560 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2svds"] Jul 05 13:42:59.661531 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.661517 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.665110 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.665088 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 13:42:59.665236 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.665092 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 13:42:59.665236 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.665183 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 13:42:59.665236 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.665193 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jq2lc\"" Jul 05 13:42:59.666253 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.666230 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 13:42:59.675604 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.675587 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2svds"] Jul 05 13:42:59.684189 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.684171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:59.747378 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.747361 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:42:59.775383 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.775364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-data-volume\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.775458 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.775395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.775458 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.775432 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.775549 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.775469 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4kw4\" (UniqueName: \"kubernetes.io/projected/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-api-access-g4kw4\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.775549 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.775531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-crio-socket\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.876353 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.876330 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4kw4\" (UniqueName: \"kubernetes.io/projected/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-api-access-g4kw4\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.876442 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.876370 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-crio-socket\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.876442 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.876418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-data-volume\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.876530 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.876446 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.876530 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.876480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.876530 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.876482 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-crio-socket\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.876632 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:42:59.876543 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:42:59.876632 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:42:59.876566 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:42:59.876632 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:42:59.876610 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:00.376595438 +0000 UTC m=+256.556437863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : configmap "kube-rbac-proxy" not found Jul 05 13:42:59.876632 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:42:59.876624 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:00.376617605 +0000 UTC m=+256.556460029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : secret "insights-runtime-extractor-tls" not found Jul 05 13:42:59.876790 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.876667 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-data-volume\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:42:59.887289 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:42:59.887263 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4kw4\" (UniqueName: \"kubernetes.io/projected/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-api-access-g4kw4\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:00.163118 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:00.163090 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:00.262426 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:00.262398 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:00.373765 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:00.373740 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:00.380300 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:00.380279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:00.380383 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:00.380358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:00.380446 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:00.380412 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:43:00.380496 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:00.380446 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:43:00.380496 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:00.380492 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:01.380474322 +0000 UTC m=+257.560316764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : configmap "kube-rbac-proxy" not found Jul 05 13:43:00.380600 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:00.380513 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:01.380502726 +0000 UTC m=+257.560345157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : secret "insights-runtime-extractor-tls" not found Jul 05 13:43:00.854213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:00.854187 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:00.925397 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:00.925374 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:01.225164 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:01.225103 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:01.322612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:01.322589 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:01.388492 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:01.388465 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:01.388623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:01.388553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:01.388623 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:01.388576 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:43:01.388741 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:01.388627 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:03.388613908 +0000 UTC m=+259.568456332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : configmap "kube-rbac-proxy" not found Jul 05 13:43:01.388741 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:01.388667 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:43:01.388741 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:01.388733 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:03.388716499 +0000 UTC m=+259.568558939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : secret "insights-runtime-extractor-tls" not found Jul 05 13:43:01.720881 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:01.720852 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r7c9v_d1f48763-233d-4180-9ae8-cd2e5934ef3b/dns-node-resolver/0.log" Jul 05 13:43:02.521901 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:02.521878 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6lmkh_43d2952f-50ec-4ce3-9239-0d5d69d5ded0/node-ca/0.log" Jul 05 13:43:02.549884 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:02.549861 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:02.653624 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:02.653599 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:03.375084 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:03.375054 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:03.404148 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:03.404125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:03.404228 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:03.404165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:03.404305 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:03.404292 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:43:03.404379 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:03.404364 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:43:03.404414 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:03.404380 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:07.404362581 +0000 UTC m=+263.584205015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : secret "insights-runtime-extractor-tls" not found Jul 05 13:43:03.404454 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:03.404424 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:07.404410727 +0000 UTC m=+263.584253155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : configmap "kube-rbac-proxy" not found Jul 05 13:43:03.522038 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:03.522018 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hgq7h_8a5d4105-f02a-4be3-aafa-e7edb24c1163/serve-healthcheck-canary/0.log" Jul 05 13:43:04.584734 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:04.584701 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:04.781854 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:04.781822 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:04.883680 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:04.883606 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:04.981914 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:04.981887 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:05.273249 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:05.273171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:05.950034 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:05.950001 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:06.270535 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:06.270436 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:06.674933 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:06.674900 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:06.871525 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:06.871489 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:07.071921 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:07.071857 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:07.431156 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:07.431125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:07.431156 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:07.431172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:07.431399 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:07.431263 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:43:07.431399 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:07.431273 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:43:07.431399 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:07.431328 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:15.43130021 +0000 UTC m=+271.611142634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : configmap "kube-rbac-proxy" not found Jul 05 13:43:07.431399 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:07.431341 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls podName:58ebfbf3-7b88-4db4-ba06-6a6c69d388ea nodeName:}" failed. No retries permitted until 2026-07-05 13:43:15.431334814 +0000 UTC m=+271.611177237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2svds" (UID: "58ebfbf3-7b88-4db4-ba06-6a6c69d388ea") : secret "insights-runtime-extractor-tls" not found Jul 05 13:43:07.480983 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:07.480951 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:08.290648 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:08.290613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:08.356855 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:08.356823 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:08.470304 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:08.470279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:08.523155 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:08.523129 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:08.872719 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:08.872685 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:09.071219 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:09.071190 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:09.527241 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:09.527209 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:10.075339 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:10.075298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:10.470642 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:10.470562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:10.672850 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:10.672820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:11.063847 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:11.063819 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:11.114718 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:11.114689 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:11.476544 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:11.476515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:12.269627 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:12.269593 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:12.464451 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:12.464420 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:12.668477 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:12.668455 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:12.870577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:12.870546 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:12.972145 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:12.972067 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:13.876242 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:13.876213 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:14.267704 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:14.267633 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:14.476160 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:14.476125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:14.653994 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:14.653969 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:14.709329 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:14.709282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:15.069827 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.069746 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:15.281476 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.281447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:15.491169 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.491146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:15.491348 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.491183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:15.491746 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.491728 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:15.493357 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.493341 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ebfbf3-7b88-4db4-ba06-6a6c69d388ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2svds\" (UID: \"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea\") " pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:15.569520 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.569494 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2svds" Jul 05 13:43:15.690485 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:15.690457 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2svds"] Jul 05 13:43:15.693118 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:15.693092 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ebfbf3_7b88_4db4_ba06_6a6c69d388ea.slice/crio-f01bb6ed243ceef8b70e4dfba06b93ffa1ff4e73b68fbb4387ed5cadd24ba7e2 WatchSource:0}: Error finding container f01bb6ed243ceef8b70e4dfba06b93ffa1ff4e73b68fbb4387ed5cadd24ba7e2: Status 404 returned error can't find the container with id f01bb6ed243ceef8b70e4dfba06b93ffa1ff4e73b68fbb4387ed5cadd24ba7e2 Jul 05 13:43:16.413485 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:16.413371 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2svds" event={"ID":"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea","Type":"ContainerStarted","Data":"f01bb6ed243ceef8b70e4dfba06b93ffa1ff4e73b68fbb4387ed5cadd24ba7e2"} Jul 05 13:43:19.281596 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:19.281572 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:19.383661 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:19.383627 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:19.424119 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:19.424094 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2svds" event={"ID":"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea","Type":"ContainerStarted","Data":"a6f5793d903d0df238fb187fc4695e678452e8772e71d9b75f1b38212acdf9b0"} Jul 05 13:43:20.429980 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:20.429940 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2svds" event={"ID":"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea","Type":"ContainerStarted","Data":"cd9f80a71080db024425596622f5cb56789eb7b4fccfa67dd9cc1f7b07c198d2"} Jul 05 13:43:21.718986 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:21.718935 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:21.819806 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:21.819774 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:21.919433 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:21.919399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:22.223105 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:22.223076 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:22.358579 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:22.358549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:22.441067 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:22.441036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2svds" event={"ID":"58ebfbf3-7b88-4db4-ba06-6a6c69d388ea","Type":"ContainerStarted","Data":"234cade1cbd8cfdb4d8a195ad169a367e511364ef96733d5d6bf7ef55f3566fa"} Jul 05 13:43:22.476485 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:22.476408 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:22.566672 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:22.566624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:23.434106 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:23.434070 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:23.614027 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:23.613997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:23.817889 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:23.817820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:24.169598 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:24.169568 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:28.098645 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:28.098588 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-77lql" podUID="3908baf2-7deb-42c1-8ccb-71a7a3008b0d" Jul 05 13:43:28.458303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:28.458229 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-77lql" Jul 05 13:43:29.930459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:29.930424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:30.084516 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:30.084479 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:30.424978 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:30.424949 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:30.996085 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:30.996036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:43:30.998409 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:30.998383 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3908baf2-7deb-42c1-8ccb-71a7a3008b0d-metrics-tls\") pod \"dns-default-77lql\" (UID: \"3908baf2-7deb-42c1-8ccb-71a7a3008b0d\") " pod="openshift-dns/dns-default-77lql" Jul 05 13:43:31.043893 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.043833 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2svds" podStartSLOduration=25.580123696 podStartE2EDuration="32.043818773s" totalImagesPullingTime="6.463695077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:42:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:22.470743624 +0000 UTC m=+278.650586069" watchObservedRunningTime="2026-07-05 13:43:31.043818773 +0000 UTC m=+287.223661218" Jul 05 13:43:31.044382 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.044368 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cf4d7996f-5wdhp"] Jul 05 13:43:31.047376 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.047361 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.051444 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051420 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 05 13:43:31.051581 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051462 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 05 13:43:31.051581 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051495 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 05 13:43:31.051581 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051526 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8z5r9\"" Jul 05 13:43:31.051581 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051536 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 05 13:43:31.051581 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051553 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 05 13:43:31.051581 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051462 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 05 13:43:31.051870 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.051808 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 05 13:43:31.059847 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.059827 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf4d7996f-5wdhp"] Jul 05 13:43:31.161946 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.161911 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sw56z\"" Jul 05 13:43:31.169204 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.169177 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-77lql" Jul 05 13:43:31.197450 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.197410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-oauth-serving-cert\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.197607 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.197466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-service-ca\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.197607 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.197489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-serving-cert\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.197607 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.197542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-config\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.197607 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.197584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2v6z\" (UniqueName: \"kubernetes.io/projected/d5cb3f6c-cb6b-4759-8589-75314fb139ee-kube-api-access-h2v6z\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.197607 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.197605 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-oauth-config\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.289300 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.289222 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-77lql"] Jul 05 13:43:31.292423 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:31.292393 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3908baf2_7deb_42c1_8ccb_71a7a3008b0d.slice/crio-ea6af0392998451ef03789fae596a6b0921032050f063c94eec0c42acd681183 WatchSource:0}: Error finding container ea6af0392998451ef03789fae596a6b0921032050f063c94eec0c42acd681183: Status 404 returned error can't find the container with id ea6af0392998451ef03789fae596a6b0921032050f063c94eec0c42acd681183 Jul 05 13:43:31.298005 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.297979 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-config\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298101 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2v6z\" (UniqueName: \"kubernetes.io/projected/d5cb3f6c-cb6b-4759-8589-75314fb139ee-kube-api-access-h2v6z\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298101 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298046 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-oauth-config\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298203 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-oauth-serving-cert\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298343 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-service-ca\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298407 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-serving-cert\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298695 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-oauth-serving-cert\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298780 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298742 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-config\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.298862 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.298784 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-service-ca\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.300579 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.300553 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-oauth-config\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.300700 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.300686 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-serving-cert\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.305897 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.305872 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2v6z\" (UniqueName: \"kubernetes.io/projected/d5cb3f6c-cb6b-4759-8589-75314fb139ee-kube-api-access-h2v6z\") pod \"console-5cf4d7996f-5wdhp\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.356886 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.356856 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:31.468408 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.468375 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-77lql" event={"ID":"3908baf2-7deb-42c1-8ccb-71a7a3008b0d","Type":"ContainerStarted","Data":"ea6af0392998451ef03789fae596a6b0921032050f063c94eec0c42acd681183"} Jul 05 13:43:31.480544 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:31.480525 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf4d7996f-5wdhp"] Jul 05 13:43:31.482525 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:31.482502 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cb3f6c_cb6b_4759_8589_75314fb139ee.slice/crio-9c449a2607fda85459fb472adf8b3a4c8e0ce300033856d86426747ab3e06cb6 WatchSource:0}: Error finding container 9c449a2607fda85459fb472adf8b3a4c8e0ce300033856d86426747ab3e06cb6: Status 404 returned error can't find the container with id 9c449a2607fda85459fb472adf8b3a4c8e0ce300033856d86426747ab3e06cb6 Jul 05 13:43:32.473621 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:32.473550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf4d7996f-5wdhp" event={"ID":"d5cb3f6c-cb6b-4759-8589-75314fb139ee","Type":"ContainerStarted","Data":"9c449a2607fda85459fb472adf8b3a4c8e0ce300033856d86426747ab3e06cb6"} Jul 05 13:43:33.478785 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:33.478736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-77lql" event={"ID":"3908baf2-7deb-42c1-8ccb-71a7a3008b0d","Type":"ContainerStarted","Data":"c0aa05f7753fae13e319c4024a78dddbfbda00aa97e9de08ed05e85e379222e5"} Jul 05 13:43:33.479273 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:33.478804 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-77lql" event={"ID":"3908baf2-7deb-42c1-8ccb-71a7a3008b0d","Type":"ContainerStarted","Data":"2eb7671255801c49b8102e7089ac765ec576e64a3c412627aba4e55e1c693316"} Jul 05 13:43:33.479273 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:33.478878 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-77lql" Jul 05 13:43:33.495982 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:33.495925 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-77lql" podStartSLOduration=250.988789474 podStartE2EDuration="4m12.495905246s" totalImagesPullingTime="1.507115772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:39:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:33.495227431 +0000 UTC m=+289.675069877" watchObservedRunningTime="2026-07-05 13:43:33.495905246 +0000 UTC m=+289.675747694" Jul 05 13:43:34.431278 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:34.431251 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:34.484111 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:34.484069 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf4d7996f-5wdhp" event={"ID":"d5cb3f6c-cb6b-4759-8589-75314fb139ee","Type":"ContainerStarted","Data":"6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3"} Jul 05 13:43:34.519798 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:34.519747 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cf4d7996f-5wdhp" podStartSLOduration=0.628383187 podStartE2EDuration="3.519728829s" totalImagesPullingTime="2.891345642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:34.51763478 +0000 UTC m=+290.697477227" watchObservedRunningTime="2026-07-05 13:43:34.519728829 +0000 UTC m=+290.699571277" Jul 05 13:43:34.530440 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:34.530414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:38.038722 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:38.038683 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:39.341692 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:39.341659 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.092555 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.092524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.165911 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.165881 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.247194 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.247166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.345266 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.345184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.357132 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.357110 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:41.357226 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.357179 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:41.361942 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.361924 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:41.411623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.411601 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.511898 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.511872 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:43:41.688137 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.688050 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.749860 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.749834 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:41.851355 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:41.851305 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:42.587295 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:42.587252 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-n66xq" podUID="c424c3fa-89b7-43e4-80e9-a09466ee2d89" Jul 05 13:43:42.944518 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:42.944441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:43.043407 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:43.043375 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:43.175029 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:43.174998 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:43.241100 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:43.241015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:43.486978 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:43.486949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-77lql" Jul 05 13:43:43.647708 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:43.647671 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:43.774860 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:43.774821 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:44.084158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:44.084127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:44.187865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:44.187834 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:44.286005 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:44.285975 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:45.068491 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.068467 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:43:45.069153 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.069134 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:43:45.071506 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.071488 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 13:43:45.252171 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.252133 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:45.480356 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.480300 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:45.532404 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.532374 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gmzwk"] Jul 05 13:43:45.535421 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.535403 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.538242 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.538211 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 13:43:45.538694 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.538469 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 13:43:45.538912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.538881 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 13:43:45.539271 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.539247 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 13:43:45.540364 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.540082 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 13:43:45.540364 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.540162 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 13:43:45.540517 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.540468 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2nddw\"" Jul 05 13:43:45.581114 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.581071 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:45.677242 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.677095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:45.705805 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.705773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-sys\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.705955 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.705809 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbe060a4-5912-4815-89f3-a18b6ea064c7-metrics-client-ca\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.705955 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.705842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.705955 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.705903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-textfile\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.706106 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.705958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.706106 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.706029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-wtmp\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.706106 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.706056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cd7w\" (UniqueName: \"kubernetes.io/projected/fbe060a4-5912-4815-89f3-a18b6ea064c7-kube-api-access-5cd7w\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.706216 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.706108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-root\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.706216 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.706126 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-tls\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.778057 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.777973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:45.806603 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806575 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-sys\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.806769 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbe060a4-5912-4815-89f3-a18b6ea064c7-metrics-client-ca\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.806769 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806640 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.806769 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806680 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-textfile\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.806769 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806683 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-sys\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.806769 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.806769 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806747 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-wtmp\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cd7w\" (UniqueName: \"kubernetes.io/projected/fbe060a4-5912-4815-89f3-a18b6ea064c7-kube-api-access-5cd7w\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-root\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806865 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-tls\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.806957 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-wtmp\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807071 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:45.806964 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 05 13:43:45.807071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.807002 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbe060a4-5912-4815-89f3-a18b6ea064c7-root\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807071 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:43:45.807041 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-tls podName:fbe060a4-5912-4815-89f3-a18b6ea064c7 nodeName:}" failed. No retries permitted until 2026-07-05 13:43:46.307017587 +0000 UTC m=+302.486860017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-tls") pod "node-exporter-gmzwk" (UID: "fbe060a4-5912-4815-89f3-a18b6ea064c7") : secret "node-exporter-tls" not found Jul 05 13:43:45.807435 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.807138 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-textfile\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807435 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.807340 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.807435 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.807341 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbe060a4-5912-4815-89f3-a18b6ea064c7-metrics-client-ca\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.809032 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.809015 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.815528 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.815504 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cd7w\" (UniqueName: \"kubernetes.io/projected/fbe060a4-5912-4815-89f3-a18b6ea064c7-kube-api-access-5cd7w\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:45.874058 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.874026 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:45.962987 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:45.962950 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:46.018450 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.018417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:46.120697 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.120667 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:46.310614 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.310576 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-tls\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:46.312820 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.312801 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbe060a4-5912-4815-89f3-a18b6ea064c7-node-exporter-tls\") pod \"node-exporter-gmzwk\" (UID: \"fbe060a4-5912-4815-89f3-a18b6ea064c7\") " pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:46.448106 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.448036 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gmzwk" Jul 05 13:43:46.457971 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:46.457937 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe060a4_5912_4815_89f3_a18b6ea064c7.slice/crio-71c8fe83ddd2d96fd2f59203bd19a18d02962a4412ea821cdb8031076c5e6a9f WatchSource:0}: Error finding container 71c8fe83ddd2d96fd2f59203bd19a18d02962a4412ea821cdb8031076c5e6a9f: Status 404 returned error can't find the container with id 71c8fe83ddd2d96fd2f59203bd19a18d02962a4412ea821cdb8031076c5e6a9f Jul 05 13:43:46.463474 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.463453 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:46.522471 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.522439 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gmzwk" event={"ID":"fbe060a4-5912-4815-89f3-a18b6ea064c7","Type":"ContainerStarted","Data":"71c8fe83ddd2d96fd2f59203bd19a18d02962a4412ea821cdb8031076c5e6a9f"} Jul 05 13:43:46.559902 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.559867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:46.568006 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.567980 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:43:46.574678 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.574658 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.580417 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.580395 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 05 13:43:46.580717 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.580692 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fqq78\"" Jul 05 13:43:46.581596 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.581575 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 05 13:43:46.581714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.581626 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 05 13:43:46.581714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.581635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 05 13:43:46.581714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.581681 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 05 13:43:46.581714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.581635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 05 13:43:46.581714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.581705 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 05 13:43:46.581972 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.581842 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 05 13:43:46.587189 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.587170 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 05 13:43:46.595263 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.595240 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:43:46.714888 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.714791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.714888 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.714838 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.714888 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.714862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.714918 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.714958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.714978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.715003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-config-out\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.715026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.715046 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.715105 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn2q4\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-kube-api-access-xn2q4\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.715146 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715475 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.715177 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-web-config\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.715475 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.715234 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.785579 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.785545 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:46.816110 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-config-out\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816110 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816111 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816161 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn2q4\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-kube-api-access-xn2q4\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-web-config\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816253 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816384 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816380 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816825 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816406 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816825 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816825 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.816825 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.816642 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.817988 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.817939 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.818167 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.818143 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.819302 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.819277 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-config-out\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.819822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.819500 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.819822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.819580 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-web-config\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.819822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.819589 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.819822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.819749 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.820060 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.819846 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.820060 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.819928 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.820722 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.820702 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.820908 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.820890 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.825074 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.825055 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn2q4\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-kube-api-access-xn2q4\") pod \"alertmanager-main-0\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:46.842209 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.842179 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:46.883905 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:46.883874 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:43:47.032143 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:47.032106 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:43:47.036019 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:47.035988 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc07b545_6310_4c77_b574_846985da4e1a.slice/crio-1d911e549e3e14fdd5054aa645eba29da08992d894e1e9693b9f978432187fce WatchSource:0}: Error finding container 1d911e549e3e14fdd5054aa645eba29da08992d894e1e9693b9f978432187fce: Status 404 returned error can't find the container with id 1d911e549e3e14fdd5054aa645eba29da08992d894e1e9693b9f978432187fce Jul 05 13:43:47.038641 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:47.038613 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:43:47.533157 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:47.533125 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerStarted","Data":"1d911e549e3e14fdd5054aa645eba29da08992d894e1e9693b9f978432187fce"} Jul 05 13:43:48.402953 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.402924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:48.487701 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.487619 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:48.537728 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.537696 2560 generic.go:356] "Generic (PLEG): container finished" podID="fbe060a4-5912-4815-89f3-a18b6ea064c7" containerID="7a2ca172e1ff812294dc5f89e468715b43833970d3d83bc62762e13c7baf0ed9" exitCode=0 Jul 05 13:43:48.538144 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.537779 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gmzwk" event={"ID":"fbe060a4-5912-4815-89f3-a18b6ea064c7","Type":"ContainerDied","Data":"7a2ca172e1ff812294dc5f89e468715b43833970d3d83bc62762e13c7baf0ed9"} Jul 05 13:43:48.539379 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.539357 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc07b545-6310-4c77-b574-846985da4e1a" containerID="bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc" exitCode=0 Jul 05 13:43:48.539452 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.539398 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc"} Jul 05 13:43:48.583806 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.583785 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:48.657196 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:48.657174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:49.287048 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.287014 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:49.388772 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.388734 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:49.545586 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.545520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gmzwk" event={"ID":"fbe060a4-5912-4815-89f3-a18b6ea064c7","Type":"ContainerStarted","Data":"65e4c826eff7c333787bc9a953ed7288ac00226ef8be21c2cf75c814108a7532"} Jul 05 13:43:49.545586 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.545566 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gmzwk" event={"ID":"fbe060a4-5912-4815-89f3-a18b6ea064c7","Type":"ContainerStarted","Data":"aeb58f6a26d45f017fd4676b9f26af0e8542c5e210035378f12bdc12ce797baf"} Jul 05 13:43:49.569714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.569657 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gmzwk" podStartSLOduration=3.489005036 podStartE2EDuration="4.569635344s" totalImagesPullingTime="1.016718982s" totalInitContainerRuntime="63.911326ms" isStatefulPod=true podCreationTimestamp="2026-07-05 13:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:49.568899082 +0000 UTC m=+305.748741527" watchObservedRunningTime="2026-07-05 13:43:49.569635344 +0000 UTC m=+305.749477791" Jul 05 13:43:49.900393 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.900361 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d6b77c859-zcqsj"] Jul 05 13:43:49.903637 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.903622 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:49.906508 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.906488 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-l97cd\"" Jul 05 13:43:49.906597 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.906522 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 05 13:43:49.906597 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.906562 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 13:43:49.907887 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.907867 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eli73ecic9r3h\"" Jul 05 13:43:49.908026 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.908009 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 05 13:43:49.908071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.908009 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 05 13:43:49.913123 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.913102 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d6b77c859-zcqsj"] Jul 05 13:43:49.994674 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:49.994653 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf4d7996f-5wdhp"] Jul 05 13:43:50.044505 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.044477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-secret-metrics-server-tls\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.044583 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.044516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bd1b475b-07a5-40b3-ba48-ac526830b507-metrics-server-audit-profiles\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.044583 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.044537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-client-ca-bundle\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.044697 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.044631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntmxl\" (UniqueName: \"kubernetes.io/projected/bd1b475b-07a5-40b3-ba48-ac526830b507-kube-api-access-ntmxl\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.044697 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.044666 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-secret-metrics-server-client-certs\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.044802 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.044696 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd1b475b-07a5-40b3-ba48-ac526830b507-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.044802 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.044745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bd1b475b-07a5-40b3-ba48-ac526830b507-audit-log\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.120060 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.120032 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:50.145111 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145089 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-secret-metrics-server-tls\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.145176 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145128 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bd1b475b-07a5-40b3-ba48-ac526830b507-metrics-server-audit-profiles\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.145176 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-client-ca-bundle\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.145264 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntmxl\" (UniqueName: \"kubernetes.io/projected/bd1b475b-07a5-40b3-ba48-ac526830b507-kube-api-access-ntmxl\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.145264 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145226 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-secret-metrics-server-client-certs\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.145264 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd1b475b-07a5-40b3-ba48-ac526830b507-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.145447 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bd1b475b-07a5-40b3-ba48-ac526830b507-audit-log\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.145900 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.145740 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bd1b475b-07a5-40b3-ba48-ac526830b507-audit-log\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.146233 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.146205 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bd1b475b-07a5-40b3-ba48-ac526830b507-metrics-server-audit-profiles\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.146480 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.146457 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd1b475b-07a5-40b3-ba48-ac526830b507-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.147472 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.147455 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-secret-metrics-server-tls\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.148096 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.148078 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-client-ca-bundle\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.148247 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.148225 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bd1b475b-07a5-40b3-ba48-ac526830b507-secret-metrics-server-client-certs\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.157072 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.157018 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntmxl\" (UniqueName: \"kubernetes.io/projected/bd1b475b-07a5-40b3-ba48-ac526830b507-kube-api-access-ntmxl\") pod \"metrics-server-6d6b77c859-zcqsj\" (UID: \"bd1b475b-07a5-40b3-ba48-ac526830b507\") " pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.213404 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.213383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:43:50.218022 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.217996 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:50.301364 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.301339 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:50.340422 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.340386 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d6b77c859-zcqsj"] Jul 05 13:43:50.344032 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:50.343998 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1b475b_07a5_40b3_ba48_ac526830b507.slice/crio-f7e55f7bf248c207261e13e8a095024a5f1d3163b4620a9cc4be9fe40099dda1 WatchSource:0}: Error finding container f7e55f7bf248c207261e13e8a095024a5f1d3163b4620a9cc4be9fe40099dda1: Status 404 returned error can't find the container with id f7e55f7bf248c207261e13e8a095024a5f1d3163b4620a9cc4be9fe40099dda1 Jul 05 13:43:50.416979 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.416895 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:50.518119 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.518073 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:50.552064 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.552036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerStarted","Data":"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8"} Jul 05 13:43:50.552471 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.552098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerStarted","Data":"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101"} Jul 05 13:43:50.552471 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.552137 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerStarted","Data":"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468"} Jul 05 13:43:50.552471 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.552163 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerStarted","Data":"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa"} Jul 05 13:43:50.552471 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.552187 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerStarted","Data":"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4"} Jul 05 13:43:50.553171 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:50.553152 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" event={"ID":"bd1b475b-07a5-40b3-ba48-ac526830b507","Type":"ContainerStarted","Data":"f7e55f7bf248c207261e13e8a095024a5f1d3163b4620a9cc4be9fe40099dda1"} Jul 05 13:43:51.451728 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.451649 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-855974555c-8ngnq"] Jul 05 13:43:51.455062 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.455043 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.464524 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.464504 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 05 13:43:51.479931 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.479894 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855974555c-8ngnq"] Jul 05 13:43:51.556623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.556593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-trusted-ca-bundle\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.556623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.556631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-oauth-serving-cert\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.557188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.556741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-oauth-config\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.557188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.556779 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz522\" (UniqueName: \"kubernetes.io/projected/2e89d8fa-7abe-4a81-992b-d2868eb27641-kube-api-access-fz522\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.557188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.556841 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-service-ca\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.557188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.556861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-serving-cert\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.557188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.556880 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-config\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.561628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.561600 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerStarted","Data":"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b"} Jul 05 13:43:51.592473 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.592421 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.13913283 podStartE2EDuration="5.592404802s" totalImagesPullingTime="3.432960952s" totalInitContainerRuntime="20.31102ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:43:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:51.590401054 +0000 UTC m=+307.770243499" watchObservedRunningTime="2026-07-05 13:43:51.592404802 +0000 UTC m=+307.772247246" Jul 05 13:43:51.657382 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.657345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-serving-cert\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.657533 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.657516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-config\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.657655 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.657632 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-trusted-ca-bundle\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.657801 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.657673 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-oauth-serving-cert\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.658670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.658241 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-oauth-config\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.658670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.658286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz522\" (UniqueName: \"kubernetes.io/projected/2e89d8fa-7abe-4a81-992b-d2868eb27641-kube-api-access-fz522\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.658670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.658344 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-oauth-serving-cert\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.658670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.658444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-service-ca\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.658670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.658560 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-config\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.658670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.658605 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-trusted-ca-bundle\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.659344 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.659304 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-service-ca\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.660786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.660766 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-serving-cert\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.661206 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.661186 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-oauth-config\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.670066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.670041 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz522\" (UniqueName: \"kubernetes.io/projected/2e89d8fa-7abe-4a81-992b-d2868eb27641-kube-api-access-fz522\") pod \"console-855974555c-8ngnq\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.697595 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.697542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:51.766350 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.766326 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:43:51.902594 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.902564 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855974555c-8ngnq"] Jul 05 13:43:51.904895 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:51.904871 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e89d8fa_7abe_4a81_992b_d2868eb27641.slice/crio-d690706bad5b1e8ffa4d81efd19c75435e2df9e30567d74c7a032d75ad47bb64 WatchSource:0}: Error finding container d690706bad5b1e8ffa4d81efd19c75435e2df9e30567d74c7a032d75ad47bb64: Status 404 returned error can't find the container with id d690706bad5b1e8ffa4d81efd19c75435e2df9e30567d74c7a032d75ad47bb64 Jul 05 13:43:51.960839 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:51.960816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:52.245347 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:52.245285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:52.301075 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:52.301044 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:52.566372 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:52.566286 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855974555c-8ngnq" event={"ID":"2e89d8fa-7abe-4a81-992b-d2868eb27641","Type":"ContainerStarted","Data":"7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb"} Jul 05 13:43:52.566791 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:52.566384 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855974555c-8ngnq" event={"ID":"2e89d8fa-7abe-4a81-992b-d2868eb27641","Type":"ContainerStarted","Data":"d690706bad5b1e8ffa4d81efd19c75435e2df9e30567d74c7a032d75ad47bb64"} Jul 05 13:43:52.567721 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:52.567696 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" event={"ID":"bd1b475b-07a5-40b3-ba48-ac526830b507","Type":"ContainerStarted","Data":"bce58999cf02253805e78851ff1c30e661e8dc58ec56941c098d93acc25ae0af"} Jul 05 13:43:52.585123 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:52.585077 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-855974555c-8ngnq" podStartSLOduration=1.5850650160000002 podStartE2EDuration="1.585065016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:52.584298869 +0000 UTC m=+308.764141317" watchObservedRunningTime="2026-07-05 13:43:52.585065016 +0000 UTC m=+308.764907463" Jul 05 13:43:52.601454 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:52.601413 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" podStartSLOduration=2.208360174 podStartE2EDuration="3.60140124s" totalImagesPullingTime="1.393041066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:52.601012599 +0000 UTC m=+308.780855045" watchObservedRunningTime="2026-07-05 13:43:52.60140124 +0000 UTC m=+308.781243686" Jul 05 13:43:56.583614 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:56.583578 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:56.585646 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:56.585626 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:43:56.684004 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:56.683970 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:43:56.921006 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:56.920927 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855974555c-8ngnq"] Jul 05 13:43:56.957531 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:56.957502 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-597cc8b5f8-hq7vw"] Jul 05 13:43:57.014853 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.014821 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597cc8b5f8-hq7vw"] Jul 05 13:43:57.015002 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.014913 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.100722 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.100689 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-serving-cert\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.100722 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.100727 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-oauth-serving-cert\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.100912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.100773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-oauth-config\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.100912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.100795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-config\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.100912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.100814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-service-ca\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.100912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.100863 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z59q\" (UniqueName: \"kubernetes.io/projected/bf138128-88de-41c7-bcb1-c6421e9b5a21-kube-api-access-8z59q\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.101040 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.100920 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-trusted-ca-bundle\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.202109 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.202027 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-trusted-ca-bundle\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.202109 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.202072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-serving-cert\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.202109 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.202107 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-oauth-serving-cert\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.202303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.202181 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-oauth-config\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.202303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.202215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-config\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.202303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.202245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-service-ca\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.202448 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.202366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z59q\" (UniqueName: \"kubernetes.io/projected/bf138128-88de-41c7-bcb1-c6421e9b5a21-kube-api-access-8z59q\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.203087 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.203064 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-service-ca\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.203087 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.203076 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-config\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.203256 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.203076 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-trusted-ca-bundle\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.203256 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.203078 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-oauth-serving-cert\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.204656 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.204638 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-oauth-config\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.204864 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.204842 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-serving-cert\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.210612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.210592 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z59q\" (UniqueName: \"kubernetes.io/projected/bf138128-88de-41c7-bcb1-c6421e9b5a21-kube-api-access-8z59q\") pod \"console-597cc8b5f8-hq7vw\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.323223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.323188 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:43:57.443135 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.443105 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-597cc8b5f8-hq7vw"] Jul 05 13:43:57.446069 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:43:57.446040 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf138128_88de_41c7_bcb1_c6421e9b5a21.slice/crio-a06b4121be2caa3d2a3c96ae0e37c64787069d81e0dc0fad98d9d56b402f2cb3 WatchSource:0}: Error finding container a06b4121be2caa3d2a3c96ae0e37c64787069d81e0dc0fad98d9d56b402f2cb3: Status 404 returned error can't find the container with id a06b4121be2caa3d2a3c96ae0e37c64787069d81e0dc0fad98d9d56b402f2cb3 Jul 05 13:43:57.590196 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.590160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597cc8b5f8-hq7vw" event={"ID":"bf138128-88de-41c7-bcb1-c6421e9b5a21","Type":"ContainerStarted","Data":"82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2"} Jul 05 13:43:57.590563 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.590213 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597cc8b5f8-hq7vw" event={"ID":"bf138128-88de-41c7-bcb1-c6421e9b5a21","Type":"ContainerStarted","Data":"a06b4121be2caa3d2a3c96ae0e37c64787069d81e0dc0fad98d9d56b402f2cb3"} Jul 05 13:43:57.605762 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:43:57.605714 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-597cc8b5f8-hq7vw" podStartSLOduration=1.6056995349999998 podStartE2EDuration="1.605699535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:43:57.604722353 +0000 UTC m=+313.784564832" watchObservedRunningTime="2026-07-05 13:43:57.605699535 +0000 UTC m=+313.785541981" Jul 05 13:44:01.018967 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:01.018937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:01.116158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:01.116127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:01.767362 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:01.767332 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:44:04.287732 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:04.287702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:04.388811 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:04.388781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:05.650853 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:05.650824 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:05.711036 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:05.711009 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:05.870737 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:05.870701 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" podUID="89f09c9d-737a-4183-8f45-e4147d0b832b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 05 13:44:07.324252 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:07.324221 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:44:07.324760 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:07.324348 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:44:07.328912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:07.328894 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:44:07.623862 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:07.623778 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:44:07.785716 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:07.785685 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:07.891619 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:07.891548 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:07.992533 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:07.992507 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:10.213703 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:10.213675 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:44:10.214150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:10.213740 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:44:15.016822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.016766 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5cf4d7996f-5wdhp" podUID="d5cb3f6c-cb6b-4759-8589-75314fb139ee" containerName="console" containerID="cri-o://6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3" gracePeriod=15 Jul 05 13:44:15.255239 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.255214 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:44:15.345398 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345372 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-oauth-serving-cert\") pod \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " Jul 05 13:44:15.345539 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345420 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-service-ca\") pod \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " Jul 05 13:44:15.345539 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345437 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-oauth-config\") pod \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " Jul 05 13:44:15.345539 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345467 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-config\") pod \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " Jul 05 13:44:15.345539 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345517 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-serving-cert\") pod \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " Jul 05 13:44:15.345755 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345550 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2v6z\" (UniqueName: \"kubernetes.io/projected/d5cb3f6c-cb6b-4759-8589-75314fb139ee-kube-api-access-h2v6z\") pod \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\" (UID: \"d5cb3f6c-cb6b-4759-8589-75314fb139ee\") " Jul 05 13:44:15.345823 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345793 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-service-ca" (OuterVolumeSpecName: "service-ca") pod "d5cb3f6c-cb6b-4759-8589-75314fb139ee" (UID: "d5cb3f6c-cb6b-4759-8589-75314fb139ee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:15.345878 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.345800 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d5cb3f6c-cb6b-4759-8589-75314fb139ee" (UID: "d5cb3f6c-cb6b-4759-8589-75314fb139ee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:15.346127 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.346101 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-config" (OuterVolumeSpecName: "console-config") pod "d5cb3f6c-cb6b-4759-8589-75314fb139ee" (UID: "d5cb3f6c-cb6b-4759-8589-75314fb139ee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:15.347615 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.347594 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d5cb3f6c-cb6b-4759-8589-75314fb139ee" (UID: "d5cb3f6c-cb6b-4759-8589-75314fb139ee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:44:15.347681 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.347665 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5cb3f6c-cb6b-4759-8589-75314fb139ee-kube-api-access-h2v6z" (OuterVolumeSpecName: "kube-api-access-h2v6z") pod "d5cb3f6c-cb6b-4759-8589-75314fb139ee" (UID: "d5cb3f6c-cb6b-4759-8589-75314fb139ee"). InnerVolumeSpecName "kube-api-access-h2v6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:15.347716 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.347688 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d5cb3f6c-cb6b-4759-8589-75314fb139ee" (UID: "d5cb3f6c-cb6b-4759-8589-75314fb139ee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:44:15.446519 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.446494 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:15.446519 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.446517 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2v6z\" (UniqueName: \"kubernetes.io/projected/d5cb3f6c-cb6b-4759-8589-75314fb139ee-kube-api-access-h2v6z\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:15.446669 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.446527 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-oauth-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:15.446669 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.446537 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:15.446669 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.446546 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-oauth-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:15.446669 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.446554 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cb3f6c-cb6b-4759-8589-75314fb139ee-console-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:15.647026 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.646994 2560 generic.go:356] "Generic (PLEG): container finished" podID="d5cb3f6c-cb6b-4759-8589-75314fb139ee" containerID="6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3" exitCode=0 Jul 05 13:44:15.647168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.647035 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf4d7996f-5wdhp" Jul 05 13:44:15.647168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.647056 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf4d7996f-5wdhp" event={"ID":"d5cb3f6c-cb6b-4759-8589-75314fb139ee","Type":"ContainerDied","Data":"6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3"} Jul 05 13:44:15.647168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.647096 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf4d7996f-5wdhp" event={"ID":"d5cb3f6c-cb6b-4759-8589-75314fb139ee","Type":"ContainerDied","Data":"9c449a2607fda85459fb472adf8b3a4c8e0ce300033856d86426747ab3e06cb6"} Jul 05 13:44:15.647168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.647135 2560 scope.go:117] "RemoveContainer" containerID="6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3" Jul 05 13:44:15.655684 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.655666 2560 scope.go:117] "RemoveContainer" containerID="6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3" Jul 05 13:44:15.655943 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:44:15.655920 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3\": container with ID starting with 6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3 not found: ID does not exist" containerID="6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3" Jul 05 13:44:15.655991 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.655952 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3"} err="failed to get container status \"6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3\": rpc error: code = NotFound desc = could not find container \"6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3\": container with ID starting with 6f5f639df8122cffbcf37aa1cd8dc673773b266db6b715ecbc7993ce13635ba3 not found: ID does not exist" Jul 05 13:44:15.662630 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.662611 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf4d7996f-5wdhp"] Jul 05 13:44:15.666303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.666284 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cf4d7996f-5wdhp"] Jul 05 13:44:15.870012 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:15.869939 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" podUID="89f09c9d-737a-4183-8f45-e4147d0b832b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 05 13:44:17.589422 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:17.589391 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5cb3f6c-cb6b-4759-8589-75314fb139ee" path="/var/lib/kubelet/pods/d5cb3f6c-cb6b-4759-8589-75314fb139ee/volumes" Jul 05 13:44:19.290468 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:19.290433 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:19.344628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:19.344603 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:21.939992 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:21.939954 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-855974555c-8ngnq" podUID="2e89d8fa-7abe-4a81-992b-d2868eb27641" containerName="console" containerID="cri-o://7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb" gracePeriod=15 Jul 05 13:44:22.175182 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.175158 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:44:22.294444 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294377 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-trusted-ca-bundle\") pod \"2e89d8fa-7abe-4a81-992b-d2868eb27641\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " Jul 05 13:44:22.294444 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294412 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-oauth-config\") pod \"2e89d8fa-7abe-4a81-992b-d2868eb27641\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " Jul 05 13:44:22.294628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294454 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz522\" (UniqueName: \"kubernetes.io/projected/2e89d8fa-7abe-4a81-992b-d2868eb27641-kube-api-access-fz522\") pod \"2e89d8fa-7abe-4a81-992b-d2868eb27641\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " Jul 05 13:44:22.294628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294492 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-serving-cert\") pod \"2e89d8fa-7abe-4a81-992b-d2868eb27641\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " Jul 05 13:44:22.294628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294532 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-service-ca\") pod \"2e89d8fa-7abe-4a81-992b-d2868eb27641\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " Jul 05 13:44:22.294628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294571 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-config\") pod \"2e89d8fa-7abe-4a81-992b-d2868eb27641\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " Jul 05 13:44:22.294628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294601 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-oauth-serving-cert\") pod \"2e89d8fa-7abe-4a81-992b-d2868eb27641\" (UID: \"2e89d8fa-7abe-4a81-992b-d2868eb27641\") " Jul 05 13:44:22.294858 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294764 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2e89d8fa-7abe-4a81-992b-d2868eb27641" (UID: "2e89d8fa-7abe-4a81-992b-d2868eb27641"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:22.294915 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294869 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-trusted-ca-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:22.294993 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294967 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-service-ca" (OuterVolumeSpecName: "service-ca") pod "2e89d8fa-7abe-4a81-992b-d2868eb27641" (UID: "2e89d8fa-7abe-4a81-992b-d2868eb27641"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:22.295051 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.294976 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-config" (OuterVolumeSpecName: "console-config") pod "2e89d8fa-7abe-4a81-992b-d2868eb27641" (UID: "2e89d8fa-7abe-4a81-992b-d2868eb27641"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:22.295051 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.295016 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2e89d8fa-7abe-4a81-992b-d2868eb27641" (UID: "2e89d8fa-7abe-4a81-992b-d2868eb27641"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:44:22.296606 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.296581 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2e89d8fa-7abe-4a81-992b-d2868eb27641" (UID: "2e89d8fa-7abe-4a81-992b-d2868eb27641"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:44:22.296726 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.296659 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e89d8fa-7abe-4a81-992b-d2868eb27641-kube-api-access-fz522" (OuterVolumeSpecName: "kube-api-access-fz522") pod "2e89d8fa-7abe-4a81-992b-d2868eb27641" (UID: "2e89d8fa-7abe-4a81-992b-d2868eb27641"). InnerVolumeSpecName "kube-api-access-fz522". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:44:22.296726 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.296687 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2e89d8fa-7abe-4a81-992b-d2868eb27641" (UID: "2e89d8fa-7abe-4a81-992b-d2868eb27641"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:44:22.396167 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.396144 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-oauth-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:22.396167 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.396167 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz522\" (UniqueName: \"kubernetes.io/projected/2e89d8fa-7abe-4a81-992b-d2868eb27641-kube-api-access-fz522\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:22.396304 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.396182 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:22.396304 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.396195 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:22.396304 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.396208 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-console-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:22.396304 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.396221 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e89d8fa-7abe-4a81-992b-d2868eb27641-oauth-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:44:22.671322 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.671277 2560 generic.go:356] "Generic (PLEG): container finished" podID="2e89d8fa-7abe-4a81-992b-d2868eb27641" containerID="7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb" exitCode=0 Jul 05 13:44:22.671484 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.671342 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855974555c-8ngnq" Jul 05 13:44:22.671484 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.671363 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855974555c-8ngnq" event={"ID":"2e89d8fa-7abe-4a81-992b-d2868eb27641","Type":"ContainerDied","Data":"7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb"} Jul 05 13:44:22.671484 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.671421 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855974555c-8ngnq" event={"ID":"2e89d8fa-7abe-4a81-992b-d2868eb27641","Type":"ContainerDied","Data":"d690706bad5b1e8ffa4d81efd19c75435e2df9e30567d74c7a032d75ad47bb64"} Jul 05 13:44:22.671484 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.671448 2560 scope.go:117] "RemoveContainer" containerID="7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb" Jul 05 13:44:22.679810 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.679791 2560 scope.go:117] "RemoveContainer" containerID="7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb" Jul 05 13:44:22.680060 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:44:22.680042 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb\": container with ID starting with 7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb not found: ID does not exist" containerID="7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb" Jul 05 13:44:22.680111 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.680068 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb"} err="failed to get container status \"7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb\": rpc error: code = NotFound desc = could not find container \"7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb\": container with ID starting with 7f55bddbc1a0c5fa574f47e6ff06095c3f0785c5be1aadc582621d49a9ebc1cb not found: ID does not exist" Jul 05 13:44:22.692486 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.692464 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855974555c-8ngnq"] Jul 05 13:44:22.696425 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:22.696405 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-855974555c-8ngnq"] Jul 05 13:44:23.593494 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:23.590764 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e89d8fa-7abe-4a81-992b-d2868eb27641" path="/var/lib/kubelet/pods/2e89d8fa-7abe-4a81-992b-d2868eb27641/volumes" Jul 05 13:44:24.043667 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:24.043584 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:24.145157 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:24.145130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:24.467644 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:24.467618 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:24.522666 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:24.522640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:25.001921 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:25.001893 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:25.103166 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:25.103134 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:25.870550 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:25.870515 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" podUID="89f09c9d-737a-4183-8f45-e4147d0b832b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 05 13:44:25.870707 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:25.870592 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" Jul 05 13:44:25.871119 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:25.871100 2560 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"6ef340bf7e218ffb3a500f989e0fbb00d2eb87edff0eab1da6b1ecc662619678"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 05 13:44:25.871156 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:25.871142 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" podUID="89f09c9d-737a-4183-8f45-e4147d0b832b" containerName="service-proxy" containerID="cri-o://6ef340bf7e218ffb3a500f989e0fbb00d2eb87edff0eab1da6b1ecc662619678" gracePeriod=30 Jul 05 13:44:26.686776 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:26.686744 2560 generic.go:356] "Generic (PLEG): container finished" podID="89f09c9d-737a-4183-8f45-e4147d0b832b" containerID="6ef340bf7e218ffb3a500f989e0fbb00d2eb87edff0eab1da6b1ecc662619678" exitCode=2 Jul 05 13:44:26.687240 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:26.686814 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" event={"ID":"89f09c9d-737a-4183-8f45-e4147d0b832b","Type":"ContainerDied","Data":"6ef340bf7e218ffb3a500f989e0fbb00d2eb87edff0eab1da6b1ecc662619678"} Jul 05 13:44:26.687240 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:26.686865 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8459f56944-5nggd" event={"ID":"89f09c9d-737a-4183-8f45-e4147d0b832b","Type":"ContainerStarted","Data":"ea6108ec7a34b5435ff73d3908d062fb33a88df157a5ee65e622ab7026952202"} Jul 05 13:44:30.218659 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:30.218633 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:44:30.222321 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:30.222290 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d6b77c859-zcqsj" Jul 05 13:44:49.290465 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:49.290424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:49.344399 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:49.344378 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:58.454924 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:58.454882 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:58.512559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:58.512530 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:44:58.955387 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:58.955360 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:44:58.957687 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:58.957662 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c424c3fa-89b7-43e4-80e9-a09466ee2d89-metrics-certs\") pod \"network-metrics-daemon-n66xq\" (UID: \"c424c3fa-89b7-43e4-80e9-a09466ee2d89\") " pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:44:58.989822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:58.989801 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fvggw\"" Jul 05 13:44:58.997051 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:58.997031 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n66xq" Jul 05 13:44:59.119940 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:59.119911 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n66xq"] Jul 05 13:44:59.122968 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:44:59.122940 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc424c3fa_89b7_43e4_80e9_a09466ee2d89.slice/crio-6877b4829b9ee4bb222164605d3330b10a9b6109be2b3330f34372ce0e299806 WatchSource:0}: Error finding container 6877b4829b9ee4bb222164605d3330b10a9b6109be2b3330f34372ce0e299806: Status 404 returned error can't find the container with id 6877b4829b9ee4bb222164605d3330b10a9b6109be2b3330f34372ce0e299806 Jul 05 13:44:59.806623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:44:59.806581 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n66xq" event={"ID":"c424c3fa-89b7-43e4-80e9-a09466ee2d89","Type":"ContainerStarted","Data":"6877b4829b9ee4bb222164605d3330b10a9b6109be2b3330f34372ce0e299806"} Jul 05 13:45:00.811716 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:00.811679 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n66xq" event={"ID":"c424c3fa-89b7-43e4-80e9-a09466ee2d89","Type":"ContainerStarted","Data":"6f71200a9dba2e0063926f1d3e526ca7006eae349fb1790b54ee7939dea32eeb"} Jul 05 13:45:00.812141 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:00.811753 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n66xq" event={"ID":"c424c3fa-89b7-43e4-80e9-a09466ee2d89","Type":"ContainerStarted","Data":"d45a0a9224da81ba0261dde658ea0e9ff6bfeb0fd3ed062cac48bd8a3568a84a"} Jul 05 13:45:00.927259 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:00.927230 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:00.995094 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:00.995069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:01.585870 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:01.585833 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:01.647470 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:01.647440 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:05.843939 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.843891 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n66xq" podStartSLOduration=379.018250999 podStartE2EDuration="6m19.84387531s" totalImagesPullingTime="825.624311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:45:00.832803939 +0000 UTC m=+377.012646398" watchObservedRunningTime="2026-07-05 13:45:05.84387531 +0000 UTC m=+382.023717737" Jul 05 13:45:05.845051 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.845032 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:45:05.845325 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.845287 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="alertmanager" containerID="cri-o://afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4" gracePeriod=120 Jul 05 13:45:05.845402 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.845366 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-metric" containerID="cri-o://c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8" gracePeriod=120 Jul 05 13:45:05.845456 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.845391 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="prom-label-proxy" containerID="cri-o://5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b" gracePeriod=120 Jul 05 13:45:05.845508 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.845437 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy" containerID="cri-o://9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101" gracePeriod=120 Jul 05 13:45:05.845508 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.845397 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-web" containerID="cri-o://675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468" gracePeriod=120 Jul 05 13:45:05.845602 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:05.845437 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="config-reloader" containerID="cri-o://b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa" gracePeriod=120 Jul 05 13:45:06.835066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835036 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc07b545-6310-4c77-b574-846985da4e1a" containerID="5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b" exitCode=0 Jul 05 13:45:06.835066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835060 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc07b545-6310-4c77-b574-846985da4e1a" containerID="9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101" exitCode=0 Jul 05 13:45:06.835066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835068 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc07b545-6310-4c77-b574-846985da4e1a" containerID="b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa" exitCode=0 Jul 05 13:45:06.835066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835074 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc07b545-6310-4c77-b574-846985da4e1a" containerID="afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4" exitCode=0 Jul 05 13:45:06.835348 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835115 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b"} Jul 05 13:45:06.835348 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101"} Jul 05 13:45:06.835348 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835210 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa"} Jul 05 13:45:06.835348 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:06.835234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4"} Jul 05 13:45:07.089802 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.089752 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.113110 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113083 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-metrics-client-ca\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113118 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-trusted-ca-bundle\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113151 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113177 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-main-tls\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113205 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113234 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn2q4\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-kube-api-access-xn2q4\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113263 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-web-config\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113287 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-config-volume\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113329 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-main-db\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113354 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-tls-assets\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113393 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-config-out\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113459 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113446 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-cluster-tls-config\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113488 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"dc07b545-6310-4c77-b574-846985da4e1a\" (UID: \"dc07b545-6310-4c77-b574-846985da4e1a\") " Jul 05 13:45:07.113786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113503 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:45:07.113786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113527 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:45:07.113786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113686 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-metrics-client-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.113786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113705 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.114090 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.113847 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:45:07.116297 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.116262 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-kube-api-access-xn2q4" (OuterVolumeSpecName: "kube-api-access-xn2q4") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "kube-api-access-xn2q4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:45:07.116624 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.116593 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:07.116727 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.116707 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:07.117131 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.117030 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:07.117131 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.117101 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-config-out" (OuterVolumeSpecName: "config-out") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:45:07.117385 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.117362 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:07.117641 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.117620 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-config-volume" (OuterVolumeSpecName: "config-volume") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:07.117709 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.117635 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:45:07.120764 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.120470 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:07.126997 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.126975 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-web-config" (OuterVolumeSpecName: "web-config") pod "dc07b545-6310-4c77-b574-846985da4e1a" (UID: "dc07b545-6310-4c77-b574-846985da4e1a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:07.214629 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214605 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-config-out\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214629 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214630 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-cluster-tls-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214646 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214660 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214674 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-main-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214688 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214703 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn2q4\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-kube-api-access-xn2q4\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214715 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-web-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214727 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc07b545-6310-4c77-b574-846985da4e1a-config-volume\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214739 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc07b545-6310-4c77-b574-846985da4e1a-alertmanager-main-db\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.214787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.214752 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc07b545-6310-4c77-b574-846985da4e1a-tls-assets\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:07.842045 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.842011 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc07b545-6310-4c77-b574-846985da4e1a" containerID="c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8" exitCode=0 Jul 05 13:45:07.842045 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.842039 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc07b545-6310-4c77-b574-846985da4e1a" containerID="675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468" exitCode=0 Jul 05 13:45:07.842224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.842061 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8"} Jul 05 13:45:07.842224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.842085 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.842224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.842124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468"} Jul 05 13:45:07.842224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.842154 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc07b545-6310-4c77-b574-846985da4e1a","Type":"ContainerDied","Data":"1d911e549e3e14fdd5054aa645eba29da08992d894e1e9693b9f978432187fce"} Jul 05 13:45:07.842224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.842186 2560 scope.go:117] "RemoveContainer" containerID="5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b" Jul 05 13:45:07.850168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.850151 2560 scope.go:117] "RemoveContainer" containerID="c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8" Jul 05 13:45:07.857108 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.857090 2560 scope.go:117] "RemoveContainer" containerID="9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101" Jul 05 13:45:07.863396 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.863374 2560 scope.go:117] "RemoveContainer" containerID="675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468" Jul 05 13:45:07.865198 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.865178 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:45:07.870127 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.870110 2560 scope.go:117] "RemoveContainer" containerID="b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa" Jul 05 13:45:07.870186 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.870138 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:45:07.876437 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.876418 2560 scope.go:117] "RemoveContainer" containerID="afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4" Jul 05 13:45:07.882761 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.882747 2560 scope.go:117] "RemoveContainer" containerID="bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc" Jul 05 13:45:07.888881 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.888863 2560 scope.go:117] "RemoveContainer" containerID="5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b" Jul 05 13:45:07.889131 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:07.889114 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b\": container with ID starting with 5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b not found: ID does not exist" containerID="5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b" Jul 05 13:45:07.889177 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889140 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b"} err="failed to get container status \"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b\": rpc error: code = NotFound desc = could not find container \"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b\": container with ID starting with 5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b not found: ID does not exist" Jul 05 13:45:07.889177 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889155 2560 scope.go:117] "RemoveContainer" containerID="c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8" Jul 05 13:45:07.889406 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:07.889389 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8\": container with ID starting with c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8 not found: ID does not exist" containerID="c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8" Jul 05 13:45:07.889452 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889411 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8"} err="failed to get container status \"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8\": rpc error: code = NotFound desc = could not find container \"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8\": container with ID starting with c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8 not found: ID does not exist" Jul 05 13:45:07.889452 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889425 2560 scope.go:117] "RemoveContainer" containerID="9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101" Jul 05 13:45:07.889632 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:07.889616 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101\": container with ID starting with 9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101 not found: ID does not exist" containerID="9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101" Jul 05 13:45:07.889673 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889636 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101"} err="failed to get container status \"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101\": rpc error: code = NotFound desc = could not find container \"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101\": container with ID starting with 9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101 not found: ID does not exist" Jul 05 13:45:07.889673 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889649 2560 scope.go:117] "RemoveContainer" containerID="675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468" Jul 05 13:45:07.889871 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:07.889856 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468\": container with ID starting with 675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468 not found: ID does not exist" containerID="675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468" Jul 05 13:45:07.889917 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889873 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468"} err="failed to get container status \"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468\": rpc error: code = NotFound desc = could not find container \"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468\": container with ID starting with 675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468 not found: ID does not exist" Jul 05 13:45:07.889917 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.889885 2560 scope.go:117] "RemoveContainer" containerID="b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa" Jul 05 13:45:07.890059 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:07.890045 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa\": container with ID starting with b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa not found: ID does not exist" containerID="b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa" Jul 05 13:45:07.890093 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890062 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa"} err="failed to get container status \"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa\": rpc error: code = NotFound desc = could not find container \"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa\": container with ID starting with b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa not found: ID does not exist" Jul 05 13:45:07.890093 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890073 2560 scope.go:117] "RemoveContainer" containerID="afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4" Jul 05 13:45:07.890270 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:07.890257 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4\": container with ID starting with afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4 not found: ID does not exist" containerID="afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4" Jul 05 13:45:07.890345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890271 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4"} err="failed to get container status \"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4\": rpc error: code = NotFound desc = could not find container \"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4\": container with ID starting with afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4 not found: ID does not exist" Jul 05 13:45:07.890345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890281 2560 scope.go:117] "RemoveContainer" containerID="bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc" Jul 05 13:45:07.890518 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:07.890499 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc\": container with ID starting with bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc not found: ID does not exist" containerID="bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc" Jul 05 13:45:07.890567 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890523 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc"} err="failed to get container status \"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc\": rpc error: code = NotFound desc = could not find container \"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc\": container with ID starting with bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc not found: ID does not exist" Jul 05 13:45:07.890567 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890538 2560 scope.go:117] "RemoveContainer" containerID="5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b" Jul 05 13:45:07.890737 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890722 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b"} err="failed to get container status \"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b\": rpc error: code = NotFound desc = could not find container \"5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b\": container with ID starting with 5cc6c06eaecc6ca56a6e7f1ee98f86d6a1cba9c412cae96396f3aea30852b64b not found: ID does not exist" Jul 05 13:45:07.890777 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890737 2560 scope.go:117] "RemoveContainer" containerID="c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8" Jul 05 13:45:07.890949 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890933 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8"} err="failed to get container status \"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8\": rpc error: code = NotFound desc = could not find container \"c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8\": container with ID starting with c946b5b8e68c1a2262cf79b74b261a49a17844c027c38a089d7bbc1fee8279e8 not found: ID does not exist" Jul 05 13:45:07.890949 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.890948 2560 scope.go:117] "RemoveContainer" containerID="9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101" Jul 05 13:45:07.891165 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891146 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101"} err="failed to get container status \"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101\": rpc error: code = NotFound desc = could not find container \"9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101\": container with ID starting with 9206685426f02e13262f200900de2fc40fd1542b587b8827a8ac38da51b9b101 not found: ID does not exist" Jul 05 13:45:07.891222 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891166 2560 scope.go:117] "RemoveContainer" containerID="675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468" Jul 05 13:45:07.891408 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891392 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468"} err="failed to get container status \"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468\": rpc error: code = NotFound desc = could not find container \"675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468\": container with ID starting with 675c20827e9385cc0fa6b3f1ac704333f0406555a3b13301e6a4f12c39f81468 not found: ID does not exist" Jul 05 13:45:07.891470 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891409 2560 scope.go:117] "RemoveContainer" containerID="b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa" Jul 05 13:45:07.891589 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891574 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa"} err="failed to get container status \"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa\": rpc error: code = NotFound desc = could not find container \"b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa\": container with ID starting with b9ba424bbc06a01385dad10920d80c569a62fc33e50a7ea36949228196d00afa not found: ID does not exist" Jul 05 13:45:07.891630 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891590 2560 scope.go:117] "RemoveContainer" containerID="afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4" Jul 05 13:45:07.891774 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891758 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4"} err="failed to get container status \"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4\": rpc error: code = NotFound desc = could not find container \"afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4\": container with ID starting with afd5e84a55ab9d0c00ee6b606533934e10cc782ad6b81d354ef1d9ef4ef992c4 not found: ID does not exist" Jul 05 13:45:07.891817 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891775 2560 scope.go:117] "RemoveContainer" containerID="bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc" Jul 05 13:45:07.891934 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.891920 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc"} err="failed to get container status \"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc\": rpc error: code = NotFound desc = could not find container \"bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc\": container with ID starting with bcaf54acdd43121f319bbaad08090fd43584510de0889964766309d58a662dfc not found: ID does not exist" Jul 05 13:45:07.896948 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.896930 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:45:07.897135 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897124 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-metric" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897136 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-metric" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897146 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="alertmanager" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897151 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="alertmanager" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897158 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e89d8fa-7abe-4a81-992b-d2868eb27641" containerName="console" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897163 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e89d8fa-7abe-4a81-992b-d2868eb27641" containerName="console" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897170 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-web" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897176 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-web" Jul 05 13:45:07.897180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897181 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897187 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897198 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="config-reloader" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897204 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="config-reloader" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897212 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="init-config-reloader" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897217 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="init-config-reloader" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897227 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5cb3f6c-cb6b-4759-8589-75314fb139ee" containerName="console" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897233 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5cb3f6c-cb6b-4759-8589-75314fb139ee" containerName="console" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897239 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="prom-label-proxy" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897243 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="prom-label-proxy" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897279 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-metric" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897286 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="prom-label-proxy" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897292 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5cb3f6c-cb6b-4759-8589-75314fb139ee" containerName="console" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897298 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="alertmanager" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897303 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e89d8fa-7abe-4a81-992b-d2868eb27641" containerName="console" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897324 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy-web" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897330 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="config-reloader" Jul 05 13:45:07.897463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.897335 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc07b545-6310-4c77-b574-846985da4e1a" containerName="kube-rbac-proxy" Jul 05 13:45:07.902092 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.902077 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.904881 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.904864 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 05 13:45:07.904970 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.904864 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 05 13:45:07.904970 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.904915 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 05 13:45:07.904970 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.904933 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 05 13:45:07.905225 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.905210 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 05 13:45:07.905287 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.905270 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fqq78\"" Jul 05 13:45:07.905352 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.905286 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 05 13:45:07.905352 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.905299 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 05 13:45:07.905576 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.905562 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 05 13:45:07.910202 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.910184 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 05 13:45:07.918324 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.918293 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921084 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921150 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921180 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-web-config\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921242 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bth2s\" (UniqueName: \"kubernetes.io/projected/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-kube-api-access-bth2s\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-config-out\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921422 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:07.923612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:07.921466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022401 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022365 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022440 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022461 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-web-config\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022528 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022554 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022590 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bth2s\" (UniqueName: \"kubernetes.io/projected/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-kube-api-access-bth2s\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022635 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-config-out\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.022686 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022685 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.023117 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.022869 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.023526 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.023414 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.024071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.024051 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.025737 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.025681 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-web-config\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.025737 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.025696 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.025737 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.025715 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.026093 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.026073 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.026181 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.026164 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.026294 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.026271 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.026380 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.026273 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.026523 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.026507 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-config-out\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.026989 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.026973 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.031168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.031147 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bth2s\" (UniqueName: \"kubernetes.io/projected/ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4-kube-api-access-bth2s\") pod \"alertmanager-main-0\" (UID: \"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.210525 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.210478 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 13:45:08.340609 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.340513 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 13:45:08.344369 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:45:08.344335 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9669eb_cece_4d2f_ae16_5e24c4a4d4c4.slice/crio-1a3771e9c2ff2c19cc9acff975a9ec5b95378914d842263d5e441dab23d61044 WatchSource:0}: Error finding container 1a3771e9c2ff2c19cc9acff975a9ec5b95378914d842263d5e441dab23d61044: Status 404 returned error can't find the container with id 1a3771e9c2ff2c19cc9acff975a9ec5b95378914d842263d5e441dab23d61044 Jul 05 13:45:08.846589 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.846554 2560 generic.go:356] "Generic (PLEG): container finished" podID="ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4" containerID="2d843e26ebff3d3098b4e6c13ff5efee12f83bf247f94d1b19100b96cce42955" exitCode=0 Jul 05 13:45:08.846766 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.846642 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerDied","Data":"2d843e26ebff3d3098b4e6c13ff5efee12f83bf247f94d1b19100b96cce42955"} Jul 05 13:45:08.846766 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:08.846707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerStarted","Data":"1a3771e9c2ff2c19cc9acff975a9ec5b95378914d842263d5e441dab23d61044"} Jul 05 13:45:09.591146 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.591117 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc07b545-6310-4c77-b574-846985da4e1a" path="/var/lib/kubelet/pods/dc07b545-6310-4c77-b574-846985da4e1a/volumes" Jul 05 13:45:09.854845 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.854768 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerStarted","Data":"3aae0cb7adf9f7b4b435f70bc354150468f942244a8fb855ec868ceeff24bd69"} Jul 05 13:45:09.854845 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.854843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerStarted","Data":"5d7e9b4198ac0e9e975faf468d254e862363d18e5aedd56ebd76f8626938f11d"} Jul 05 13:45:09.855053 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.854883 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerStarted","Data":"c6a71f2853d821727672bf921c847939c889d6f0aa331f90798ac5e1a34fd65a"} Jul 05 13:45:09.855053 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.854918 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerStarted","Data":"c2dee4826523654e2e4831b3cc86fbe5aace40e25a823dd503db78712157a60b"} Jul 05 13:45:09.855053 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.854953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerStarted","Data":"939314cd52fc6ce575b3ec6622688e3315da6ca9dfbab42a6ae5c7d7b6791dcd"} Jul 05 13:45:09.855053 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.854987 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4","Type":"ContainerStarted","Data":"56539b9aa03532e413c220608b5d105d38e919eae14dbf00734a877735b50c9e"} Jul 05 13:45:09.885216 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:09.885173 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.865367893 podStartE2EDuration="2.88515588s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.787987ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:45:09.88315399 +0000 UTC m=+386.062996461" watchObservedRunningTime="2026-07-05 13:45:09.88515588 +0000 UTC m=+386.064998328" Jul 05 13:45:11.474577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.474543 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6496d678d9-skfz2"] Jul 05 13:45:11.477946 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.477928 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.492767 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.492745 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6496d678d9-skfz2"] Jul 05 13:45:11.546657 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.546638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-service-ca\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.546753 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.546665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m8kp\" (UniqueName: \"kubernetes.io/projected/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-kube-api-access-8m8kp\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.546753 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.546690 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-oauth-serving-cert\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.546753 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.546711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-oauth-config\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.546872 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.546763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-serving-cert\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.546872 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.546781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-config\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.546872 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.546800 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-trusted-ca-bundle\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.559431 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.559404 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:11.616039 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.616012 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:11.647303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.647280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-oauth-config\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.647406 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.647335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-serving-cert\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.647406 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.647368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-config\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.647406 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.647395 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-trusted-ca-bundle\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.647567 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.647455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-service-ca\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.647567 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.647482 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m8kp\" (UniqueName: \"kubernetes.io/projected/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-kube-api-access-8m8kp\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.647567 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.647521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-oauth-serving-cert\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.648244 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.648218 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-config\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.648354 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.648239 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-trusted-ca-bundle\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.648354 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.648294 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-oauth-serving-cert\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.648431 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.648364 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-service-ca\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.649915 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.649884 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-oauth-config\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.650014 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.649963 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-serving-cert\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.655612 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.655594 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m8kp\" (UniqueName: \"kubernetes.io/projected/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-kube-api-access-8m8kp\") pod \"console-6496d678d9-skfz2\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.786778 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.786712 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:11.908792 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:11.908769 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6496d678d9-skfz2"] Jul 05 13:45:11.910923 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:45:11.910889 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca51ef2e_a8e3_4fdb_9402_b5a3c401e5da.slice/crio-9f8174cf6303f559af4d8aac27bd62334ec13ffbb0416138c2da03b886826784 WatchSource:0}: Error finding container 9f8174cf6303f559af4d8aac27bd62334ec13ffbb0416138c2da03b886826784: Status 404 returned error can't find the container with id 9f8174cf6303f559af4d8aac27bd62334ec13ffbb0416138c2da03b886826784 Jul 05 13:45:12.870301 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:12.870267 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6496d678d9-skfz2" event={"ID":"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da","Type":"ContainerStarted","Data":"5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b"} Jul 05 13:45:12.870700 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:12.870328 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6496d678d9-skfz2" event={"ID":"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da","Type":"ContainerStarted","Data":"9f8174cf6303f559af4d8aac27bd62334ec13ffbb0416138c2da03b886826784"} Jul 05 13:45:12.890666 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:12.890621 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6496d678d9-skfz2" podStartSLOduration=1.890606223 podStartE2EDuration="1.890606223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:45:12.889888484 +0000 UTC m=+389.069730944" watchObservedRunningTime="2026-07-05 13:45:12.890606223 +0000 UTC m=+389.070448726" Jul 05 13:45:13.660284 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:13.660254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:13.719319 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:13.719286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:19.325042 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:19.325009 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:19.417018 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:19.416988 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:21.787430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:21.787381 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:21.787851 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:21.787460 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:21.792104 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:21.792084 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:21.905030 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:21.905005 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:45:21.951871 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:21.951843 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-597cc8b5f8-hq7vw"] Jul 05 13:45:22.062225 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:22.062150 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:22.159860 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:22.159830 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:22.258966 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:22.258934 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:46.971175 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:46.971139 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-597cc8b5f8-hq7vw" podUID="bf138128-88de-41c7-bcb1-c6421e9b5a21" containerName="console" containerID="cri-o://82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2" gracePeriod=15 Jul 05 13:45:47.204672 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.204652 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:45:47.293395 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293330 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-oauth-config\") pod \"bf138128-88de-41c7-bcb1-c6421e9b5a21\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " Jul 05 13:45:47.293395 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293361 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-oauth-serving-cert\") pod \"bf138128-88de-41c7-bcb1-c6421e9b5a21\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " Jul 05 13:45:47.293537 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293400 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z59q\" (UniqueName: \"kubernetes.io/projected/bf138128-88de-41c7-bcb1-c6421e9b5a21-kube-api-access-8z59q\") pod \"bf138128-88de-41c7-bcb1-c6421e9b5a21\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " Jul 05 13:45:47.293537 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293435 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-serving-cert\") pod \"bf138128-88de-41c7-bcb1-c6421e9b5a21\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " Jul 05 13:45:47.293537 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293481 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-service-ca\") pod \"bf138128-88de-41c7-bcb1-c6421e9b5a21\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " Jul 05 13:45:47.293664 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293545 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-trusted-ca-bundle\") pod \"bf138128-88de-41c7-bcb1-c6421e9b5a21\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " Jul 05 13:45:47.293664 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293571 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-config\") pod \"bf138128-88de-41c7-bcb1-c6421e9b5a21\" (UID: \"bf138128-88de-41c7-bcb1-c6421e9b5a21\") " Jul 05 13:45:47.293828 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293801 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bf138128-88de-41c7-bcb1-c6421e9b5a21" (UID: "bf138128-88de-41c7-bcb1-c6421e9b5a21"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:45:47.294049 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.293941 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-service-ca" (OuterVolumeSpecName: "service-ca") pod "bf138128-88de-41c7-bcb1-c6421e9b5a21" (UID: "bf138128-88de-41c7-bcb1-c6421e9b5a21"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:45:47.294137 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.294107 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bf138128-88de-41c7-bcb1-c6421e9b5a21" (UID: "bf138128-88de-41c7-bcb1-c6421e9b5a21"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:45:47.294193 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.294144 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-config" (OuterVolumeSpecName: "console-config") pod "bf138128-88de-41c7-bcb1-c6421e9b5a21" (UID: "bf138128-88de-41c7-bcb1-c6421e9b5a21"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:45:47.295605 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.295578 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf138128-88de-41c7-bcb1-c6421e9b5a21-kube-api-access-8z59q" (OuterVolumeSpecName: "kube-api-access-8z59q") pod "bf138128-88de-41c7-bcb1-c6421e9b5a21" (UID: "bf138128-88de-41c7-bcb1-c6421e9b5a21"). InnerVolumeSpecName "kube-api-access-8z59q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:45:47.295944 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.295927 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bf138128-88de-41c7-bcb1-c6421e9b5a21" (UID: "bf138128-88de-41c7-bcb1-c6421e9b5a21"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:47.296014 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.295948 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bf138128-88de-41c7-bcb1-c6421e9b5a21" (UID: "bf138128-88de-41c7-bcb1-c6421e9b5a21"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:45:47.394541 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.394522 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-trusted-ca-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:47.394541 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.394543 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:47.394657 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.394552 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-oauth-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:47.394657 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.394561 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-oauth-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:47.394657 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.394570 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z59q\" (UniqueName: \"kubernetes.io/projected/bf138128-88de-41c7-bcb1-c6421e9b5a21-kube-api-access-8z59q\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:47.394657 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.394579 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf138128-88de-41c7-bcb1-c6421e9b5a21-console-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:47.394657 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.394587 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bf138128-88de-41c7-bcb1-c6421e9b5a21-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:45:47.990994 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.990908 2560 generic.go:356] "Generic (PLEG): container finished" podID="bf138128-88de-41c7-bcb1-c6421e9b5a21" containerID="82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2" exitCode=0 Jul 05 13:45:47.990994 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.990968 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-597cc8b5f8-hq7vw" Jul 05 13:45:47.991430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.990996 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597cc8b5f8-hq7vw" event={"ID":"bf138128-88de-41c7-bcb1-c6421e9b5a21","Type":"ContainerDied","Data":"82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2"} Jul 05 13:45:47.991430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.991050 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-597cc8b5f8-hq7vw" event={"ID":"bf138128-88de-41c7-bcb1-c6421e9b5a21","Type":"ContainerDied","Data":"a06b4121be2caa3d2a3c96ae0e37c64787069d81e0dc0fad98d9d56b402f2cb3"} Jul 05 13:45:47.991430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.991077 2560 scope.go:117] "RemoveContainer" containerID="82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2" Jul 05 13:45:47.999158 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.999142 2560 scope.go:117] "RemoveContainer" containerID="82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2" Jul 05 13:45:47.999457 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:45:47.999427 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2\": container with ID starting with 82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2 not found: ID does not exist" containerID="82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2" Jul 05 13:45:47.999514 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:47.999464 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2"} err="failed to get container status \"82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2\": rpc error: code = NotFound desc = could not find container \"82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2\": container with ID starting with 82b76686e715f63206b53e725c65529e8e1fac94d9e4e221c9eed7827a7f64c2 not found: ID does not exist" Jul 05 13:45:48.006966 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:48.006944 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-597cc8b5f8-hq7vw"] Jul 05 13:45:48.012915 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:48.012893 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-597cc8b5f8-hq7vw"] Jul 05 13:45:49.590159 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:49.590126 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf138128-88de-41c7-bcb1-c6421e9b5a21" path="/var/lib/kubelet/pods/bf138128-88de-41c7-bcb1-c6421e9b5a21/volumes" Jul 05 13:45:53.946038 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:53.946007 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:54.038748 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.038716 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:45:54.750053 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.750020 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr"] Jul 05 13:45:54.750299 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.750287 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf138128-88de-41c7-bcb1-c6421e9b5a21" containerName="console" Jul 05 13:45:54.750363 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.750300 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf138128-88de-41c7-bcb1-c6421e9b5a21" containerName="console" Jul 05 13:45:54.750363 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.750353 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf138128-88de-41c7-bcb1-c6421e9b5a21" containerName="console" Jul 05 13:45:54.754665 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.754649 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.757522 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.757504 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:45:54.757640 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.757505 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xmzp7\"" Jul 05 13:45:54.757640 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.757595 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:45:54.763816 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.763798 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr"] Jul 05 13:45:54.843017 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.842998 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.843142 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.843045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.843142 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.843125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5c6s\" (UniqueName: \"kubernetes.io/projected/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-kube-api-access-n5c6s\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.944319 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.944292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.944402 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.944358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5c6s\" (UniqueName: \"kubernetes.io/projected/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-kube-api-access-n5c6s\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.944402 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.944383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.944642 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.944618 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.944682 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.944642 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:54.953429 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:54.953406 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5c6s\" (UniqueName: \"kubernetes.io/projected/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-kube-api-access-n5c6s\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:55.064244 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:55.064180 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:45:55.188186 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:55.188120 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr"] Jul 05 13:45:55.190759 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:45:55.190732 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b6500f_6bdc_4dc0_ab29_04d40e7d6f4a.slice/crio-d6a95f385c056225800da6fae3df6faf25b1ef1ff7a325d68736b7ab98182abf WatchSource:0}: Error finding container d6a95f385c056225800da6fae3df6faf25b1ef1ff7a325d68736b7ab98182abf: Status 404 returned error can't find the container with id d6a95f385c056225800da6fae3df6faf25b1ef1ff7a325d68736b7ab98182abf Jul 05 13:45:56.019540 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:45:56.019505 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" event={"ID":"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a","Type":"ContainerStarted","Data":"d6a95f385c056225800da6fae3df6faf25b1ef1ff7a325d68736b7ab98182abf"} Jul 05 13:46:01.040226 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:01.040197 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" event={"ID":"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a","Type":"ContainerStarted","Data":"85d6804605fcf22f4258237023e83cffa46e38a57508f7f8244c5979832255f8"} Jul 05 13:46:01.339157 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:01.339130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:01.403298 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:01.403274 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:02.044643 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:02.044606 2560 generic.go:356] "Generic (PLEG): container finished" podID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerID="85d6804605fcf22f4258237023e83cffa46e38a57508f7f8244c5979832255f8" exitCode=0 Jul 05 13:46:02.045026 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:02.044659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" event={"ID":"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a","Type":"ContainerDied","Data":"85d6804605fcf22f4258237023e83cffa46e38a57508f7f8244c5979832255f8"} Jul 05 13:46:05.057006 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:05.056975 2560 generic.go:356] "Generic (PLEG): container finished" podID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerID="32991d4bb4db54976a3e5f667871256dfe07bd9c8f3e19e6db763213a0820d4d" exitCode=0 Jul 05 13:46:05.057428 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:05.057050 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" event={"ID":"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a","Type":"ContainerDied","Data":"32991d4bb4db54976a3e5f667871256dfe07bd9c8f3e19e6db763213a0820d4d"} Jul 05 13:46:12.087980 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:12.087945 2560 generic.go:356] "Generic (PLEG): container finished" podID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerID="4e6104145683cdf93ec2a9bac993530c5467be5777f5c0ee693a4a4b3202b3d6" exitCode=0 Jul 05 13:46:12.088387 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:12.087992 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" event={"ID":"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a","Type":"ContainerDied","Data":"4e6104145683cdf93ec2a9bac993530c5467be5777f5c0ee693a4a4b3202b3d6"} Jul 05 13:46:13.212224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.212203 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:46:13.295229 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.295208 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-bundle\") pod \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " Jul 05 13:46:13.295393 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.295250 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-util\") pod \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " Jul 05 13:46:13.295393 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.295337 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5c6s\" (UniqueName: \"kubernetes.io/projected/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-kube-api-access-n5c6s\") pod \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\" (UID: \"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a\") " Jul 05 13:46:13.295899 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.295874 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-bundle" (OuterVolumeSpecName: "bundle") pod "c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" (UID: "c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:13.297402 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.297375 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-kube-api-access-n5c6s" (OuterVolumeSpecName: "kube-api-access-n5c6s") pod "c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" (UID: "c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a"). InnerVolumeSpecName "kube-api-access-n5c6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:13.299496 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.299476 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-util" (OuterVolumeSpecName: "util") pod "c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" (UID: "c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:13.395986 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.395931 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5c6s\" (UniqueName: \"kubernetes.io/projected/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-kube-api-access-n5c6s\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:13.395986 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.395955 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:13.395986 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:13.395965 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a-util\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:14.097169 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:14.097139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" event={"ID":"c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a","Type":"ContainerDied","Data":"d6a95f385c056225800da6fae3df6faf25b1ef1ff7a325d68736b7ab98182abf"} Jul 05 13:46:14.097349 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:14.097181 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6a95f385c056225800da6fae3df6faf25b1ef1ff7a325d68736b7ab98182abf" Jul 05 13:46:14.097349 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:14.097181 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfnbr" Jul 05 13:46:15.742033 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:15.742005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:16.633199 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:16.633170 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:17.731478 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731449 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt"] Jul 05 13:46:17.731836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731669 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerName="util" Jul 05 13:46:17.731836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731680 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerName="util" Jul 05 13:46:17.731836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731691 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerName="extract" Jul 05 13:46:17.731836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731696 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerName="extract" Jul 05 13:46:17.731836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731714 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerName="pull" Jul 05 13:46:17.731836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731719 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerName="pull" Jul 05 13:46:17.731836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.731766 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1b6500f-6bdc-4dc0-ab29-04d40e7d6f4a" containerName="extract" Jul 05 13:46:17.763457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.763436 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt"] Jul 05 13:46:17.763579 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.763513 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:17.766729 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.766706 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:46:17.766853 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.766786 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 05 13:46:17.767452 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.767439 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-h2cf9\"" Jul 05 13:46:17.928373 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.928344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4z7r\" (UniqueName: \"kubernetes.io/projected/0c5b5f0c-bccf-45ed-97b6-af5933532eff-kube-api-access-j4z7r\") pod \"cert-manager-operator-controller-manager-55c444bf86-qq8kt\" (UID: \"0c5b5f0c-bccf-45ed-97b6-af5933532eff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:17.928489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:17.928381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c5b5f0c-bccf-45ed-97b6-af5933532eff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qq8kt\" (UID: \"0c5b5f0c-bccf-45ed-97b6-af5933532eff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:18.029683 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:18.029618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4z7r\" (UniqueName: \"kubernetes.io/projected/0c5b5f0c-bccf-45ed-97b6-af5933532eff-kube-api-access-j4z7r\") pod \"cert-manager-operator-controller-manager-55c444bf86-qq8kt\" (UID: \"0c5b5f0c-bccf-45ed-97b6-af5933532eff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:18.029683 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:18.029657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c5b5f0c-bccf-45ed-97b6-af5933532eff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qq8kt\" (UID: \"0c5b5f0c-bccf-45ed-97b6-af5933532eff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:18.029970 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:18.029956 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c5b5f0c-bccf-45ed-97b6-af5933532eff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qq8kt\" (UID: \"0c5b5f0c-bccf-45ed-97b6-af5933532eff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:18.039150 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:18.039133 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4z7r\" (UniqueName: \"kubernetes.io/projected/0c5b5f0c-bccf-45ed-97b6-af5933532eff-kube-api-access-j4z7r\") pod \"cert-manager-operator-controller-manager-55c444bf86-qq8kt\" (UID: \"0c5b5f0c-bccf-45ed-97b6-af5933532eff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:18.072128 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:18.072108 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" Jul 05 13:46:18.214230 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:18.214198 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt"] Jul 05 13:46:18.218156 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:46:18.218129 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5b5f0c_bccf_45ed_97b6_af5933532eff.slice/crio-428347cfa6d6563f98738efb1560dabe64c40f24c2c45e65613956c11656d679 WatchSource:0}: Error finding container 428347cfa6d6563f98738efb1560dabe64c40f24c2c45e65613956c11656d679: Status 404 returned error can't find the container with id 428347cfa6d6563f98738efb1560dabe64c40f24c2c45e65613956c11656d679 Jul 05 13:46:19.118265 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:19.118231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" event={"ID":"0c5b5f0c-bccf-45ed-97b6-af5933532eff","Type":"ContainerStarted","Data":"428347cfa6d6563f98738efb1560dabe64c40f24c2c45e65613956c11656d679"} Jul 05 13:46:19.359567 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:19.359534 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:19.420031 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:19.419968 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:21.127596 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:21.127564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" event={"ID":"0c5b5f0c-bccf-45ed-97b6-af5933532eff","Type":"ContainerStarted","Data":"803f50f8bb036a60f80d182ea4c4ae6529f8b25e5769c18a35f00fb140d61be9"} Jul 05 13:46:21.152255 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:21.152199 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qq8kt" podStartSLOduration=2.063486156 podStartE2EDuration="4.152180225s" totalImagesPullingTime="2.088694069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:46:21.150389981 +0000 UTC m=+457.330232424" watchObservedRunningTime="2026-07-05 13:46:21.152180225 +0000 UTC m=+457.332022698" Jul 05 13:46:21.157487 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:21.157466 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:21.255467 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:21.255449 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:21.554913 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:21.554846 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:21.770810 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:21.770781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:22.081943 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.081910 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd"] Jul 05 13:46:22.096782 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.096757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.104059 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.104038 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:46:22.104245 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.104038 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xmzp7\"" Jul 05 13:46:22.104245 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.104233 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:46:22.106056 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.106033 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd"] Jul 05 13:46:22.262411 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.262381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.262778 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.262448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.262778 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.262467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jml4\" (UniqueName: \"kubernetes.io/projected/8a0561b4-79a1-479b-87c3-f0f635792253-kube-api-access-4jml4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.362770 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.362703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.362770 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.362759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.362929 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.362785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jml4\" (UniqueName: \"kubernetes.io/projected/8a0561b4-79a1-479b-87c3-f0f635792253-kube-api-access-4jml4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.363058 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.363040 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.363154 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.363136 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.371662 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.371641 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jml4\" (UniqueName: \"kubernetes.io/projected/8a0561b4-79a1-479b-87c3-f0f635792253-kube-api-access-4jml4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.406745 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.406724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:22.573439 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:22.573418 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd"] Jul 05 13:46:22.575284 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:46:22.575253 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0561b4_79a1_479b_87c3_f0f635792253.slice/crio-07533b697d25a645db66a390ee425e022c0be8e25b139be808cfd940c38234ee WatchSource:0}: Error finding container 07533b697d25a645db66a390ee425e022c0be8e25b139be808cfd940c38234ee: Status 404 returned error can't find the container with id 07533b697d25a645db66a390ee425e022c0be8e25b139be808cfd940c38234ee Jul 05 13:46:23.139670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:23.139639 2560 generic.go:356] "Generic (PLEG): container finished" podID="8a0561b4-79a1-479b-87c3-f0f635792253" containerID="ba4afbab7f9f142e4161e5676d6123fe80e97be599e25d21da95246aebf5bdae" exitCode=0 Jul 05 13:46:23.139835 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:23.139718 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" event={"ID":"8a0561b4-79a1-479b-87c3-f0f635792253","Type":"ContainerDied","Data":"ba4afbab7f9f142e4161e5676d6123fe80e97be599e25d21da95246aebf5bdae"} Jul 05 13:46:23.139835 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:23.139780 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" event={"ID":"8a0561b4-79a1-479b-87c3-f0f635792253","Type":"ContainerStarted","Data":"07533b697d25a645db66a390ee425e022c0be8e25b139be808cfd940c38234ee"} Jul 05 13:46:23.184189 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:23.184166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:23.550382 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:23.550294 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:26.151844 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:26.151811 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:26.153906 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:26.153879 2560 generic.go:356] "Generic (PLEG): container finished" podID="8a0561b4-79a1-479b-87c3-f0f635792253" containerID="e4a139af4cdf6efe0978cf3d01fc007b6a93c53dac83a6108b5d244fecb42e3d" exitCode=0 Jul 05 13:46:26.154008 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:26.153953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" event={"ID":"8a0561b4-79a1-479b-87c3-f0f635792253","Type":"ContainerDied","Data":"e4a139af4cdf6efe0978cf3d01fc007b6a93c53dac83a6108b5d244fecb42e3d"} Jul 05 13:46:26.255256 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:26.255231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:27.160438 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.160403 2560 generic.go:356] "Generic (PLEG): container finished" podID="8a0561b4-79a1-479b-87c3-f0f635792253" containerID="6c61b938517f7444165acae5988c0fd29813d57d0b6be5b2f1d58ff7fdbf8e1f" exitCode=0 Jul 05 13:46:27.160794 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.160463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" event={"ID":"8a0561b4-79a1-479b-87c3-f0f635792253","Type":"ContainerDied","Data":"6c61b938517f7444165acae5988c0fd29813d57d0b6be5b2f1d58ff7fdbf8e1f"} Jul 05 13:46:27.736523 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.736491 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f9djv"] Jul 05 13:46:27.738520 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.738505 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:27.741432 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.741401 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 05 13:46:27.741524 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.741472 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 05 13:46:27.742666 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.742653 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-n8qgx\"" Jul 05 13:46:27.750483 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.750461 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f9djv"] Jul 05 13:46:27.901925 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.901897 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghwbb\" (UniqueName: \"kubernetes.io/projected/a70e6bfd-167f-4197-babd-0a47b92e67c8-kube-api-access-ghwbb\") pod \"cert-manager-cainjector-559cdd8c87-f9djv\" (UID: \"a70e6bfd-167f-4197-babd-0a47b92e67c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:27.902049 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:27.901935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a70e6bfd-167f-4197-babd-0a47b92e67c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f9djv\" (UID: \"a70e6bfd-167f-4197-babd-0a47b92e67c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:28.002793 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.002723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghwbb\" (UniqueName: \"kubernetes.io/projected/a70e6bfd-167f-4197-babd-0a47b92e67c8-kube-api-access-ghwbb\") pod \"cert-manager-cainjector-559cdd8c87-f9djv\" (UID: \"a70e6bfd-167f-4197-babd-0a47b92e67c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:28.002793 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.002770 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a70e6bfd-167f-4197-babd-0a47b92e67c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f9djv\" (UID: \"a70e6bfd-167f-4197-babd-0a47b92e67c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:28.011771 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.011751 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a70e6bfd-167f-4197-babd-0a47b92e67c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f9djv\" (UID: \"a70e6bfd-167f-4197-babd-0a47b92e67c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:28.011865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.011752 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghwbb\" (UniqueName: \"kubernetes.io/projected/a70e6bfd-167f-4197-babd-0a47b92e67c8-kube-api-access-ghwbb\") pod \"cert-manager-cainjector-559cdd8c87-f9djv\" (UID: \"a70e6bfd-167f-4197-babd-0a47b92e67c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:28.047358 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.047338 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" Jul 05 13:46:28.172823 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.172682 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f9djv"] Jul 05 13:46:28.174708 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:46:28.174678 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda70e6bfd_167f_4197_babd_0a47b92e67c8.slice/crio-307e689143bb4fb06d2eef7ff02dd9249c831bf791915641a71ee860cecb9f84 WatchSource:0}: Error finding container 307e689143bb4fb06d2eef7ff02dd9249c831bf791915641a71ee860cecb9f84: Status 404 returned error can't find the container with id 307e689143bb4fb06d2eef7ff02dd9249c831bf791915641a71ee860cecb9f84 Jul 05 13:46:28.279479 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.279460 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:28.405097 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.405069 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jml4\" (UniqueName: \"kubernetes.io/projected/8a0561b4-79a1-479b-87c3-f0f635792253-kube-api-access-4jml4\") pod \"8a0561b4-79a1-479b-87c3-f0f635792253\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " Jul 05 13:46:28.405257 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.405139 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-bundle\") pod \"8a0561b4-79a1-479b-87c3-f0f635792253\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " Jul 05 13:46:28.405257 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.405168 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-util\") pod \"8a0561b4-79a1-479b-87c3-f0f635792253\" (UID: \"8a0561b4-79a1-479b-87c3-f0f635792253\") " Jul 05 13:46:28.405568 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.405537 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-bundle" (OuterVolumeSpecName: "bundle") pod "8a0561b4-79a1-479b-87c3-f0f635792253" (UID: "8a0561b4-79a1-479b-87c3-f0f635792253"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:28.407077 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.407055 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0561b4-79a1-479b-87c3-f0f635792253-kube-api-access-4jml4" (OuterVolumeSpecName: "kube-api-access-4jml4") pod "8a0561b4-79a1-479b-87c3-f0f635792253" (UID: "8a0561b4-79a1-479b-87c3-f0f635792253"). InnerVolumeSpecName "kube-api-access-4jml4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:28.409916 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.409884 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-util" (OuterVolumeSpecName: "util") pod "8a0561b4-79a1-479b-87c3-f0f635792253" (UID: "8a0561b4-79a1-479b-87c3-f0f635792253"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:28.506284 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.506259 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:28.506284 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.506279 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a0561b4-79a1-479b-87c3-f0f635792253-util\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:28.506428 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:28.506290 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jml4\" (UniqueName: \"kubernetes.io/projected/8a0561b4-79a1-479b-87c3-f0f635792253-kube-api-access-4jml4\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:29.171635 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:29.171603 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" Jul 05 13:46:29.171820 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:29.171601 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtcjd" event={"ID":"8a0561b4-79a1-479b-87c3-f0f635792253","Type":"ContainerDied","Data":"07533b697d25a645db66a390ee425e022c0be8e25b139be808cfd940c38234ee"} Jul 05 13:46:29.171820 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:29.171729 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07533b697d25a645db66a390ee425e022c0be8e25b139be808cfd940c38234ee" Jul 05 13:46:29.172972 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:29.172935 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" event={"ID":"a70e6bfd-167f-4197-babd-0a47b92e67c8","Type":"ContainerStarted","Data":"307e689143bb4fb06d2eef7ff02dd9249c831bf791915641a71ee860cecb9f84"} Jul 05 13:46:30.144457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:30.144425 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:31.182663 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:31.182629 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" event={"ID":"a70e6bfd-167f-4197-babd-0a47b92e67c8","Type":"ContainerStarted","Data":"6ef6fd960b69069738f5c3db6206b6129575f85144286b62f34803f5db2cf454"} Jul 05 13:46:31.199659 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:31.199617 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f9djv" podStartSLOduration=1.859450276 podStartE2EDuration="4.199602135s" totalImagesPullingTime="2.340151859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:46:31.19853124 +0000 UTC m=+467.378373685" watchObservedRunningTime="2026-07-05 13:46:31.199602135 +0000 UTC m=+467.379444580" Jul 05 13:46:31.286038 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:31.286007 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:31.385837 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:31.385806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:31.953137 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:31.953107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:37.159820 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:37.159790 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:37.260179 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:37.260147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:43.386253 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:43.386220 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:43.486946 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:43.486919 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:44.831810 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.831772 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f"] Jul 05 13:46:44.832213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.832069 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0561b4-79a1-479b-87c3-f0f635792253" containerName="pull" Jul 05 13:46:44.832213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.832081 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0561b4-79a1-479b-87c3-f0f635792253" containerName="pull" Jul 05 13:46:44.832213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.832093 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0561b4-79a1-479b-87c3-f0f635792253" containerName="extract" Jul 05 13:46:44.832213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.832099 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0561b4-79a1-479b-87c3-f0f635792253" containerName="extract" Jul 05 13:46:44.832213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.832114 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0561b4-79a1-479b-87c3-f0f635792253" containerName="util" Jul 05 13:46:44.832213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.832119 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0561b4-79a1-479b-87c3-f0f635792253" containerName="util" Jul 05 13:46:44.832213 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.832193 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a0561b4-79a1-479b-87c3-f0f635792253" containerName="extract" Jul 05 13:46:44.835168 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.835152 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:44.838157 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.838135 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:46:44.839326 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.839293 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:46:44.839409 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.839353 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xmzp7\"" Jul 05 13:46:44.843422 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.843400 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f"] Jul 05 13:46:44.927974 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.927953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:44.928081 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.927986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:44.928081 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:44.928007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw68x\" (UniqueName: \"kubernetes.io/projected/cef8b5aa-6057-45ca-be1c-db4457b51a76-kube-api-access-zw68x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.029016 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.028995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.029127 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.029029 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.029127 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.029050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw68x\" (UniqueName: \"kubernetes.io/projected/cef8b5aa-6057-45ca-be1c-db4457b51a76-kube-api-access-zw68x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.029372 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.029354 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.029447 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.029388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.037965 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.037948 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 05 13:46:45.048396 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.048375 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 05 13:46:45.058071 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.058048 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw68x\" (UniqueName: \"kubernetes.io/projected/cef8b5aa-6057-45ca-be1c-db4457b51a76-kube-api-access-zw68x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.147506 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.147468 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xmzp7\"" Jul 05 13:46:45.155689 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.155671 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:45.282865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.282833 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f"] Jul 05 13:46:45.285906 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:46:45.285880 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef8b5aa_6057_45ca_be1c_db4457b51a76.slice/crio-34efa9e91956caca84d56af8656f595ec6cbf7b5491b3782dd8aa7d6408b9258 WatchSource:0}: Error finding container 34efa9e91956caca84d56af8656f595ec6cbf7b5491b3782dd8aa7d6408b9258: Status 404 returned error can't find the container with id 34efa9e91956caca84d56af8656f595ec6cbf7b5491b3782dd8aa7d6408b9258 Jul 05 13:46:45.336613 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:45.336590 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:46.245703 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:46.245673 2560 generic.go:356] "Generic (PLEG): container finished" podID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerID="4ccdfedec62d19f7cc6d1b3fdf615aae9ce3e806c1042721296fc0d6ad923d9e" exitCode=0 Jul 05 13:46:46.246038 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:46.245715 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" event={"ID":"cef8b5aa-6057-45ca-be1c-db4457b51a76","Type":"ContainerDied","Data":"4ccdfedec62d19f7cc6d1b3fdf615aae9ce3e806c1042721296fc0d6ad923d9e"} Jul 05 13:46:46.246038 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:46.245751 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" event={"ID":"cef8b5aa-6057-45ca-be1c-db4457b51a76","Type":"ContainerStarted","Data":"34efa9e91956caca84d56af8656f595ec6cbf7b5491b3782dd8aa7d6408b9258"} Jul 05 13:46:47.251797 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:47.251762 2560 generic.go:356] "Generic (PLEG): container finished" podID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerID="060cd68796774261c755c73e51f3790f9abc240a665bea9a5290fa1ee302e09f" exitCode=0 Jul 05 13:46:47.252170 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:47.251829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" event={"ID":"cef8b5aa-6057-45ca-be1c-db4457b51a76","Type":"ContainerDied","Data":"060cd68796774261c755c73e51f3790f9abc240a665bea9a5290fa1ee302e09f"} Jul 05 13:46:48.257855 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:48.257818 2560 generic.go:356] "Generic (PLEG): container finished" podID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerID="6edf44c494e1b1415f9a3c5ac61756897497e340cff481728abdc0e9aeec3e38" exitCode=0 Jul 05 13:46:48.258226 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:48.257893 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" event={"ID":"cef8b5aa-6057-45ca-be1c-db4457b51a76","Type":"ContainerDied","Data":"6edf44c494e1b1415f9a3c5ac61756897497e340cff481728abdc0e9aeec3e38"} Jul 05 13:46:49.384028 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.384004 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:49.460055 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.460025 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw68x\" (UniqueName: \"kubernetes.io/projected/cef8b5aa-6057-45ca-be1c-db4457b51a76-kube-api-access-zw68x\") pod \"cef8b5aa-6057-45ca-be1c-db4457b51a76\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " Jul 05 13:46:49.460223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.460081 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-util\") pod \"cef8b5aa-6057-45ca-be1c-db4457b51a76\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " Jul 05 13:46:49.460223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.460100 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-bundle\") pod \"cef8b5aa-6057-45ca-be1c-db4457b51a76\" (UID: \"cef8b5aa-6057-45ca-be1c-db4457b51a76\") " Jul 05 13:46:49.460815 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.460792 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-bundle" (OuterVolumeSpecName: "bundle") pod "cef8b5aa-6057-45ca-be1c-db4457b51a76" (UID: "cef8b5aa-6057-45ca-be1c-db4457b51a76"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:49.462064 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.462048 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef8b5aa-6057-45ca-be1c-db4457b51a76-kube-api-access-zw68x" (OuterVolumeSpecName: "kube-api-access-zw68x") pod "cef8b5aa-6057-45ca-be1c-db4457b51a76" (UID: "cef8b5aa-6057-45ca-be1c-db4457b51a76"). InnerVolumeSpecName "kube-api-access-zw68x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:49.468461 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.468435 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-util" (OuterVolumeSpecName: "util") pod "cef8b5aa-6057-45ca-be1c-db4457b51a76" (UID: "cef8b5aa-6057-45ca-be1c-db4457b51a76"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:49.560763 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.560689 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-util\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:49.560763 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.560722 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef8b5aa-6057-45ca-be1c-db4457b51a76-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:49.560763 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:49.560735 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw68x\" (UniqueName: \"kubernetes.io/projected/cef8b5aa-6057-45ca-be1c-db4457b51a76-kube-api-access-zw68x\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:46:50.267677 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:50.267647 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" Jul 05 13:46:50.267842 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:50.267647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx7n8f" event={"ID":"cef8b5aa-6057-45ca-be1c-db4457b51a76","Type":"ContainerDied","Data":"34efa9e91956caca84d56af8656f595ec6cbf7b5491b3782dd8aa7d6408b9258"} Jul 05 13:46:50.267842 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:50.267794 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34efa9e91956caca84d56af8656f595ec6cbf7b5491b3782dd8aa7d6408b9258" Jul 05 13:46:55.770042 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:55.770004 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:59.647775 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.647733 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:59.773696 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.773664 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:46:59.829208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829180 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j"] Jul 05 13:46:59.829436 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829424 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerName="pull" Jul 05 13:46:59.829489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829437 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerName="pull" Jul 05 13:46:59.829489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829449 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerName="util" Jul 05 13:46:59.829489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829455 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerName="util" Jul 05 13:46:59.829489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829486 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerName="extract" Jul 05 13:46:59.829489 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829491 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerName="extract" Jul 05 13:46:59.829646 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.829533 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef8b5aa-6057-45ca-be1c-db4457b51a76" containerName="extract" Jul 05 13:46:59.836490 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.836472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:46:59.839935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.839913 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 05 13:46:59.840018 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.839954 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 05 13:46:59.841180 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.841163 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 05 13:46:59.841270 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.841208 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 05 13:46:59.841353 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.841268 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:46:59.841417 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.841377 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zp774\"" Jul 05 13:46:59.850761 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.850743 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j"] Jul 05 13:46:59.929521 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.929461 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-cert\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:46:59.929521 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.929494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-manager-config\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:46:59.929656 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.929573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-metrics-cert\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:46:59.929656 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:46:59.929631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjp7j\" (UniqueName: \"kubernetes.io/projected/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-kube-api-access-qjp7j\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.030617 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.030583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-cert\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.030745 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.030627 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-manager-config\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.030745 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.030676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-metrics-cert\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.030745 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.030708 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjp7j\" (UniqueName: \"kubernetes.io/projected/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-kube-api-access-qjp7j\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.031266 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.031247 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-manager-config\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.032955 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.032939 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-cert\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.033050 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.032991 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-metrics-cert\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.054517 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.054496 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjp7j\" (UniqueName: \"kubernetes.io/projected/13394b7d-9ebc-4d76-9dd4-f5309e87d9c2-kube-api-access-qjp7j\") pod \"lws-controller-manager-bf64756c4-pk87j\" (UID: \"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2\") " pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.145354 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.145330 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:00.317474 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:00.317455 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j"] Jul 05 13:47:00.319701 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:47:00.319670 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13394b7d_9ebc_4d76_9dd4_f5309e87d9c2.slice/crio-ae640b87547c2aae62291163fab24c482f6a5afc3f3e106246dd58dfb4456fa6 WatchSource:0}: Error finding container ae640b87547c2aae62291163fab24c482f6a5afc3f3e106246dd58dfb4456fa6: Status 404 returned error can't find the container with id ae640b87547c2aae62291163fab24c482f6a5afc3f3e106246dd58dfb4456fa6 Jul 05 13:47:01.313744 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:01.313706 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" event={"ID":"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2","Type":"ContainerStarted","Data":"ae640b87547c2aae62291163fab24c482f6a5afc3f3e106246dd58dfb4456fa6"} Jul 05 13:47:01.395858 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:01.395806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:01.493812 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:01.493782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.034197 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.034093 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.305903 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.305806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.319051 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.319014 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" event={"ID":"13394b7d-9ebc-4d76-9dd4-f5309e87d9c2","Type":"ContainerStarted","Data":"a6318fcf3927cd509914f4cfeab49b0d1f93742185be9c072be3902e80e18596"} Jul 05 13:47:02.319444 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.319099 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:02.344979 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.344933 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" podStartSLOduration=1.9377564569999999 podStartE2EDuration="3.344918136s" totalImagesPullingTime="1.407161679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:47:02.344539906 +0000 UTC m=+498.524382354" watchObservedRunningTime="2026-07-05 13:47:02.344918136 +0000 UTC m=+498.524760582" Jul 05 13:47:02.403195 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.403168 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.459364 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.459336 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg"] Jul 05 13:47:02.462522 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.462504 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.466367 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.466344 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x926h\"" Jul 05 13:47:02.466556 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.466534 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 13:47:02.466708 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.466671 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 13:47:02.466846 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.466792 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 13:47:02.466961 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.466941 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 13:47:02.483474 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.483454 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg"] Jul 05 13:47:02.549672 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.549643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e49baf1a-1120-4185-931f-084ef9a6ea75-manifests\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.549838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.549696 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e49baf1a-1120-4185-931f-084ef9a6ea75-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.549838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.549774 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxb8d\" (UniqueName: \"kubernetes.io/projected/e49baf1a-1120-4185-931f-084ef9a6ea75-kube-api-access-qxb8d\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.549947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.549832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e49baf1a-1120-4185-931f-084ef9a6ea75-tmp\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.549947 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.549869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e49baf1a-1120-4185-931f-084ef9a6ea75-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.576532 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.576507 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.651092 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.651063 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e49baf1a-1120-4185-931f-084ef9a6ea75-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.651223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.651103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxb8d\" (UniqueName: \"kubernetes.io/projected/e49baf1a-1120-4185-931f-084ef9a6ea75-kube-api-access-qxb8d\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.651223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.651129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e49baf1a-1120-4185-931f-084ef9a6ea75-tmp\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.651223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.651150 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e49baf1a-1120-4185-931f-084ef9a6ea75-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.651223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.651189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e49baf1a-1120-4185-931f-084ef9a6ea75-manifests\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.651638 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.651616 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e49baf1a-1120-4185-931f-084ef9a6ea75-manifests\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.653458 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.653430 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e49baf1a-1120-4185-931f-084ef9a6ea75-tmp\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.653544 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.653481 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e49baf1a-1120-4185-931f-084ef9a6ea75-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.653584 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.653563 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e49baf1a-1120-4185-931f-084ef9a6ea75-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.659334 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.659294 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxb8d\" (UniqueName: \"kubernetes.io/projected/e49baf1a-1120-4185-931f-084ef9a6ea75-kube-api-access-qxb8d\") pod \"opendatahub-operator-controller-manager-5f79c67cfc-cpgtg\" (UID: \"e49baf1a-1120-4185-931f-084ef9a6ea75\") " pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.676243 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.676219 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.772912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.772877 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:02.783810 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.783783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.879625 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.879600 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:02.908984 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.904957 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg"] Jul 05 13:47:02.984047 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:02.984018 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:03.324296 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:03.324259 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" event={"ID":"e49baf1a-1120-4185-931f-084ef9a6ea75","Type":"ContainerStarted","Data":"a85b1f585b8ac1988b17e26213b157a043863d5a140241c0e6c683e249647643"} Jul 05 13:47:06.340793 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:06.340760 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" event={"ID":"e49baf1a-1120-4185-931f-084ef9a6ea75","Type":"ContainerStarted","Data":"c007181ae919b4fde4422c5da16f6c756f22dc485f53af6f7d70c872111f5e4b"} Jul 05 13:47:07.346068 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:07.346038 2560 generic.go:356] "Generic (PLEG): container finished" podID="e49baf1a-1120-4185-931f-084ef9a6ea75" containerID="c007181ae919b4fde4422c5da16f6c756f22dc485f53af6f7d70c872111f5e4b" exitCode=0 Jul 05 13:47:07.346510 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:07.346127 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" event={"ID":"e49baf1a-1120-4185-931f-084ef9a6ea75","Type":"ContainerDied","Data":"c007181ae919b4fde4422c5da16f6c756f22dc485f53af6f7d70c872111f5e4b"} Jul 05 13:47:08.353455 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:08.353414 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" event={"ID":"e49baf1a-1120-4185-931f-084ef9a6ea75","Type":"ContainerStarted","Data":"206b352fd0ca1eb174c2a4d2ce2a9a8e886c539d6cd1cec29e28b7fa07a94635"} Jul 05 13:47:08.353940 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:08.353549 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:08.377589 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:08.377539 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" podStartSLOduration=2.228154368 podStartE2EDuration="6.377523105s" totalImagesPullingTime="3.123348523s" totalInitContainerRuntime="1.026020214s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:47:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:47:08.376411961 +0000 UTC m=+504.556254406" watchObservedRunningTime="2026-07-05 13:47:08.377523105 +0000 UTC m=+504.557365554" Jul 05 13:47:13.326164 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:13.326135 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bf64756c4-pk87j" Jul 05 13:47:13.943369 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:13.943334 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:14.044266 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:14.044223 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:19.360291 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:19.360263 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f79c67cfc-cpgtg" Jul 05 13:47:22.618482 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:22.618442 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:22.796956 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:22.796888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:24.352085 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:24.352058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:28.542577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:28.542538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:28.734153 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:28.734116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:28.839013 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:28.838975 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:28.934882 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:28.934853 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:29.280400 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:29.280287 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:29.511446 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:29.511405 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:29.882279 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:29.882243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:29.980344 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:29.980294 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:30.881020 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:30.880982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:35.083866 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.083831 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:35.184190 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.184154 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:35.883951 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.883913 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb"] Jul 05 13:47:35.893885 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.893854 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:35.900546 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.900336 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 05 13:47:35.900546 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.900392 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 05 13:47:35.900546 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.900411 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 13:47:35.900546 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.900414 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 13:47:35.900546 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.900485 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xrlzl\"" Jul 05 13:47:35.901155 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.900612 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 05 13:47:35.901155 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.900692 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 05 13:47:35.926192 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:35.926152 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb"] Jul 05 13:47:36.013407 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.013353 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2c46a8-542d-46d0-afae-d39717b167f6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.013407 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.013411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.013647 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.013474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.013647 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.013566 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtgzz\" (UniqueName: \"kubernetes.io/projected/3e2c46a8-542d-46d0-afae-d39717b167f6-kube-api-access-vtgzz\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.013647 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.013606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.013647 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.013626 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.013784 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.013665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.114554 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.114512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2c46a8-542d-46d0-afae-d39717b167f6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.114554 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.114559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.115074 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.114687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.115074 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.114759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtgzz\" (UniqueName: \"kubernetes.io/projected/3e2c46a8-542d-46d0-afae-d39717b167f6-kube-api-access-vtgzz\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.115074 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.114810 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.115074 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.114839 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.115074 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.114873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.115660 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.115637 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.117151 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.117126 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.117269 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.117193 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2c46a8-542d-46d0-afae-d39717b167f6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.117551 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.117524 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.117714 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.117695 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.123966 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.123931 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtgzz\" (UniqueName: \"kubernetes.io/projected/3e2c46a8-542d-46d0-afae-d39717b167f6-kube-api-access-vtgzz\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.124422 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.124403 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2c46a8-542d-46d0-afae-d39717b167f6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6wvvb\" (UID: \"3e2c46a8-542d-46d0-afae-d39717b167f6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.203965 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.203859 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:36.336039 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.336014 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb"] Jul 05 13:47:36.338483 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:47:36.338454 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2c46a8_542d_46d0_afae_d39717b167f6.slice/crio-033a960abcf208ceca4b7fb206f1810134f81d1c76968e70186958e99cdc2003 WatchSource:0}: Error finding container 033a960abcf208ceca4b7fb206f1810134f81d1c76968e70186958e99cdc2003: Status 404 returned error can't find the container with id 033a960abcf208ceca4b7fb206f1810134f81d1c76968e70186958e99cdc2003 Jul 05 13:47:36.477563 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.477464 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" event={"ID":"3e2c46a8-542d-46d0-afae-d39717b167f6","Type":"ContainerStarted","Data":"033a960abcf208ceca4b7fb206f1810134f81d1c76968e70186958e99cdc2003"} Jul 05 13:47:36.962254 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:36.962199 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:37.098357 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:37.098295 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:38.947879 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:38.947843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:39.058760 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:39.058717 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:39.156943 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:39.156918 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:39.228860 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:39.228820 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 05 13:47:39.228971 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:39.228889 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 05 13:47:39.275966 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:39.275942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:39.493061 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:39.492968 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" event={"ID":"3e2c46a8-542d-46d0-afae-d39717b167f6","Type":"ContainerStarted","Data":"8e8dfc35605c4d14a01ee1567508665e8fe99afe9df66ecbea6d2aa35c7683f1"} Jul 05 13:47:39.493226 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:39.493081 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:40.240136 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.240103 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:40.364710 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.364674 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:40.499231 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.499162 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" Jul 05 13:47:40.511275 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.511249 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:40.571300 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.571247 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6wvvb" podStartSLOduration=2.683105145 podStartE2EDuration="5.571229803s" totalImagesPullingTime="2.888124658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:47:39.518442046 +0000 UTC m=+535.698284493" watchObservedRunningTime="2026-07-05 13:47:40.571229803 +0000 UTC m=+536.751072252" Jul 05 13:47:40.737228 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.737177 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:40.880530 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.880492 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:47:40.988986 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:47:40.988947 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:09.202733 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:09.202698 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:09.304644 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:09.304608 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:24.025251 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:24.025214 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:24.118535 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:24.118503 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:24.218729 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:24.218693 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:48:39.516364 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.516263 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st"] Jul 05 13:48:39.519463 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.519447 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.522519 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.522498 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 13:48:39.523682 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.523645 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 13:48:39.523782 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.523707 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ljxll\"" Jul 05 13:48:39.529503 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.529481 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st"] Jul 05 13:48:39.617076 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.617050 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.617183 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.617084 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.617183 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.617118 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zpmj\" (UniqueName: \"kubernetes.io/projected/296dc6e5-d09d-46ae-ade5-8f448bd240d6-kube-api-access-5zpmj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.717641 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.717615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.717641 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.717643 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.717784 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.717665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zpmj\" (UniqueName: \"kubernetes.io/projected/296dc6e5-d09d-46ae-ade5-8f448bd240d6-kube-api-access-5zpmj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.717980 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.717963 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.718022 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.717998 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.726802 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.726779 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zpmj\" (UniqueName: \"kubernetes.io/projected/296dc6e5-d09d-46ae-ade5-8f448bd240d6-kube-api-access-5zpmj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.828391 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.828371 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:39.953421 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:39.953393 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st"] Jul 05 13:48:39.955302 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:48:39.955273 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296dc6e5_d09d_46ae_ade5_8f448bd240d6.slice/crio-1b6a8f5ad8bdae165e30c319447c313f1b50afabaa2e62aac81413a9d95ceeba WatchSource:0}: Error finding container 1b6a8f5ad8bdae165e30c319447c313f1b50afabaa2e62aac81413a9d95ceeba: Status 404 returned error can't find the container with id 1b6a8f5ad8bdae165e30c319447c313f1b50afabaa2e62aac81413a9d95ceeba Jul 05 13:48:40.114416 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.114350 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h"] Jul 05 13:48:40.117480 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.117460 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.127341 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.127300 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h"] Jul 05 13:48:40.222248 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.222222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7tv4\" (UniqueName: \"kubernetes.io/projected/2eec340b-e114-4fdd-b435-7e118b909439-kube-api-access-r7tv4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.222414 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.222258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.222414 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.222274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.323521 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.323489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7tv4\" (UniqueName: \"kubernetes.io/projected/2eec340b-e114-4fdd-b435-7e118b909439-kube-api-access-r7tv4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.323641 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.323534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.323641 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.323559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.323938 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.323921 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.323977 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.323939 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.333325 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.333286 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7tv4\" (UniqueName: \"kubernetes.io/projected/2eec340b-e114-4fdd-b435-7e118b909439-kube-api-access-r7tv4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.426296 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.426230 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:40.511765 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.511727 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv"] Jul 05 13:48:40.517265 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.517244 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.526485 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.526458 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv"] Jul 05 13:48:40.553949 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.553920 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h"] Jul 05 13:48:40.556043 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:48:40.556020 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eec340b_e114_4fdd_b435_7e118b909439.slice/crio-99f7ea2d19ecf1a9cea7a12da87b6791975b0c4a43985c0d9e2e0f6346278a72 WatchSource:0}: Error finding container 99f7ea2d19ecf1a9cea7a12da87b6791975b0c4a43985c0d9e2e0f6346278a72: Status 404 returned error can't find the container with id 99f7ea2d19ecf1a9cea7a12da87b6791975b0c4a43985c0d9e2e0f6346278a72 Jul 05 13:48:40.625516 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.625493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.625618 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.625524 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkpsn\" (UniqueName: \"kubernetes.io/projected/b883cab1-a89b-435e-b330-68119697e3d1-kube-api-access-kkpsn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.625618 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.625597 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.726859 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.726799 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.726859 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.726837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkpsn\" (UniqueName: \"kubernetes.io/projected/b883cab1-a89b-435e-b330-68119697e3d1-kube-api-access-kkpsn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.727021 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.726928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.727240 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.727219 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.727331 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.727221 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.736574 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.736553 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkpsn\" (UniqueName: \"kubernetes.io/projected/b883cab1-a89b-435e-b330-68119697e3d1-kube-api-access-kkpsn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.746791 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.746770 2560 generic.go:356] "Generic (PLEG): container finished" podID="2eec340b-e114-4fdd-b435-7e118b909439" containerID="77a7d28957398ca1e7ba54a2da7d9275e7114dc17cfa731489b3b5a376d81b6b" exitCode=0 Jul 05 13:48:40.746888 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.746841 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" event={"ID":"2eec340b-e114-4fdd-b435-7e118b909439","Type":"ContainerDied","Data":"77a7d28957398ca1e7ba54a2da7d9275e7114dc17cfa731489b3b5a376d81b6b"} Jul 05 13:48:40.746888 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.746878 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" event={"ID":"2eec340b-e114-4fdd-b435-7e118b909439","Type":"ContainerStarted","Data":"99f7ea2d19ecf1a9cea7a12da87b6791975b0c4a43985c0d9e2e0f6346278a72"} Jul 05 13:48:40.748422 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.748375 2560 generic.go:356] "Generic (PLEG): container finished" podID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerID="eae7409b9ff6b0c86db153e159a20cfa7fc1a82a71dec29606465894efb34a9f" exitCode=0 Jul 05 13:48:40.748505 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.748454 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" event={"ID":"296dc6e5-d09d-46ae-ade5-8f448bd240d6","Type":"ContainerDied","Data":"eae7409b9ff6b0c86db153e159a20cfa7fc1a82a71dec29606465894efb34a9f"} Jul 05 13:48:40.748505 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.748492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" event={"ID":"296dc6e5-d09d-46ae-ade5-8f448bd240d6","Type":"ContainerStarted","Data":"1b6a8f5ad8bdae165e30c319447c313f1b50afabaa2e62aac81413a9d95ceeba"} Jul 05 13:48:40.829242 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.829220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:40.916967 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.916937 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn"] Jul 05 13:48:40.920614 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.920593 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:40.930215 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.930099 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn"] Jul 05 13:48:40.964018 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:40.963928 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv"] Jul 05 13:48:40.965974 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:48:40.965949 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb883cab1_a89b_435e_b330_68119697e3d1.slice/crio-7d5eb3eb0d664d6f0adebd0b5e08d6c9fd6e5030b568e9da32f785b2cfa9bd57 WatchSource:0}: Error finding container 7d5eb3eb0d664d6f0adebd0b5e08d6c9fd6e5030b568e9da32f785b2cfa9bd57: Status 404 returned error can't find the container with id 7d5eb3eb0d664d6f0adebd0b5e08d6c9fd6e5030b568e9da32f785b2cfa9bd57 Jul 05 13:48:41.031695 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.031666 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.031816 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.031797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhxr2\" (UniqueName: \"kubernetes.io/projected/28c22f75-7e29-42ea-8a78-032e925944fe-kube-api-access-fhxr2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.031861 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.031827 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.132701 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.132672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhxr2\" (UniqueName: \"kubernetes.io/projected/28c22f75-7e29-42ea-8a78-032e925944fe-kube-api-access-fhxr2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.132819 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.132711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.132819 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.132752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.133089 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.133070 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.133156 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.133105 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.142033 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.142014 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhxr2\" (UniqueName: \"kubernetes.io/projected/28c22f75-7e29-42ea-8a78-032e925944fe-kube-api-access-fhxr2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.233755 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.233711 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:41.360342 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.360296 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn"] Jul 05 13:48:41.367364 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:48:41.367333 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c22f75_7e29_42ea_8a78_032e925944fe.slice/crio-c668fc0c5328b12ad28f5d5586db75cf480282d7ee692976c3bde6f757f09fac WatchSource:0}: Error finding container c668fc0c5328b12ad28f5d5586db75cf480282d7ee692976c3bde6f757f09fac: Status 404 returned error can't find the container with id c668fc0c5328b12ad28f5d5586db75cf480282d7ee692976c3bde6f757f09fac Jul 05 13:48:41.755084 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.755049 2560 generic.go:356] "Generic (PLEG): container finished" podID="b883cab1-a89b-435e-b330-68119697e3d1" containerID="0e64ce859165662b088cde1f14df2f6862d090fa0e02dc7d3de85ef54e29b3d1" exitCode=0 Jul 05 13:48:41.755451 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.755110 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" event={"ID":"b883cab1-a89b-435e-b330-68119697e3d1","Type":"ContainerDied","Data":"0e64ce859165662b088cde1f14df2f6862d090fa0e02dc7d3de85ef54e29b3d1"} Jul 05 13:48:41.755451 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.755162 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" event={"ID":"b883cab1-a89b-435e-b330-68119697e3d1","Type":"ContainerStarted","Data":"7d5eb3eb0d664d6f0adebd0b5e08d6c9fd6e5030b568e9da32f785b2cfa9bd57"} Jul 05 13:48:41.756758 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.756735 2560 generic.go:356] "Generic (PLEG): container finished" podID="28c22f75-7e29-42ea-8a78-032e925944fe" containerID="03b95c830e24d90d50b100296dddb7455e63dc6825b5d1770248f8a7dd7add7f" exitCode=0 Jul 05 13:48:41.756867 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.756822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" event={"ID":"28c22f75-7e29-42ea-8a78-032e925944fe","Type":"ContainerDied","Data":"03b95c830e24d90d50b100296dddb7455e63dc6825b5d1770248f8a7dd7add7f"} Jul 05 13:48:41.756924 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.756869 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" event={"ID":"28c22f75-7e29-42ea-8a78-032e925944fe","Type":"ContainerStarted","Data":"c668fc0c5328b12ad28f5d5586db75cf480282d7ee692976c3bde6f757f09fac"} Jul 05 13:48:41.758618 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.758600 2560 generic.go:356] "Generic (PLEG): container finished" podID="2eec340b-e114-4fdd-b435-7e118b909439" containerID="975830144c8afff11407a15b94462bb2850273bacad3eaa8a1bf772233a9fae7" exitCode=0 Jul 05 13:48:41.758702 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.758623 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" event={"ID":"2eec340b-e114-4fdd-b435-7e118b909439","Type":"ContainerDied","Data":"975830144c8afff11407a15b94462bb2850273bacad3eaa8a1bf772233a9fae7"} Jul 05 13:48:41.760733 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.760709 2560 generic.go:356] "Generic (PLEG): container finished" podID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerID="dbf7fa2f296eca6367a774d7019e26f68c7ea0b6e2bf4f169ad5bc6d3892a446" exitCode=0 Jul 05 13:48:41.760843 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:41.760795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" event={"ID":"296dc6e5-d09d-46ae-ade5-8f448bd240d6","Type":"ContainerDied","Data":"dbf7fa2f296eca6367a774d7019e26f68c7ea0b6e2bf4f169ad5bc6d3892a446"} Jul 05 13:48:42.769765 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.769736 2560 generic.go:356] "Generic (PLEG): container finished" podID="b883cab1-a89b-435e-b330-68119697e3d1" containerID="adb39fa69ce0cb38f461ecc0db44915c543bd89ea0ec372e6e201be6e58ebeec" exitCode=0 Jul 05 13:48:42.770120 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.769820 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" event={"ID":"b883cab1-a89b-435e-b330-68119697e3d1","Type":"ContainerDied","Data":"adb39fa69ce0cb38f461ecc0db44915c543bd89ea0ec372e6e201be6e58ebeec"} Jul 05 13:48:42.771651 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.771628 2560 generic.go:356] "Generic (PLEG): container finished" podID="28c22f75-7e29-42ea-8a78-032e925944fe" containerID="3ea9fa75b38c065c41c185f7ce81cd60f4dc1a9182ad0e16e8403ca38b7731a8" exitCode=0 Jul 05 13:48:42.771744 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.771711 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" event={"ID":"28c22f75-7e29-42ea-8a78-032e925944fe","Type":"ContainerDied","Data":"3ea9fa75b38c065c41c185f7ce81cd60f4dc1a9182ad0e16e8403ca38b7731a8"} Jul 05 13:48:42.773786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.773769 2560 generic.go:356] "Generic (PLEG): container finished" podID="2eec340b-e114-4fdd-b435-7e118b909439" containerID="f891105f9fa7971dfc76e9f8fe60e2578cad355f8b747db65f88461c2879b02d" exitCode=0 Jul 05 13:48:42.773873 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.773834 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" event={"ID":"2eec340b-e114-4fdd-b435-7e118b909439","Type":"ContainerDied","Data":"f891105f9fa7971dfc76e9f8fe60e2578cad355f8b747db65f88461c2879b02d"} Jul 05 13:48:42.776143 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.776100 2560 generic.go:356] "Generic (PLEG): container finished" podID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerID="5adfaa8ffee9f3bd72d66438110eb5af48957b62dabd3947d0adac560db1e725" exitCode=0 Jul 05 13:48:42.776143 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:42.776135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" event={"ID":"296dc6e5-d09d-46ae-ade5-8f448bd240d6","Type":"ContainerDied","Data":"5adfaa8ffee9f3bd72d66438110eb5af48957b62dabd3947d0adac560db1e725"} Jul 05 13:48:43.782559 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:43.782523 2560 generic.go:356] "Generic (PLEG): container finished" podID="b883cab1-a89b-435e-b330-68119697e3d1" containerID="0a29a2f118b47ecbf893d1b8380a6be1a43922515689268185a3754c01e7af40" exitCode=0 Jul 05 13:48:43.782971 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:43.782612 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" event={"ID":"b883cab1-a89b-435e-b330-68119697e3d1","Type":"ContainerDied","Data":"0a29a2f118b47ecbf893d1b8380a6be1a43922515689268185a3754c01e7af40"} Jul 05 13:48:43.784630 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:43.784609 2560 generic.go:356] "Generic (PLEG): container finished" podID="28c22f75-7e29-42ea-8a78-032e925944fe" containerID="b085a737b359f6560e4230b49bf07d9eb9cc40e96961bfa32c9280b992e448a4" exitCode=0 Jul 05 13:48:43.784734 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:43.784700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" event={"ID":"28c22f75-7e29-42ea-8a78-032e925944fe","Type":"ContainerDied","Data":"b085a737b359f6560e4230b49bf07d9eb9cc40e96961bfa32c9280b992e448a4"} Jul 05 13:48:43.931683 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:43.931658 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:43.934373 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:43.934352 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:44.058020 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.057945 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-util\") pod \"2eec340b-e114-4fdd-b435-7e118b909439\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " Jul 05 13:48:44.058020 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.058006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-bundle\") pod \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " Jul 05 13:48:44.058291 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.058042 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-util\") pod \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " Jul 05 13:48:44.058291 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.058078 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zpmj\" (UniqueName: \"kubernetes.io/projected/296dc6e5-d09d-46ae-ade5-8f448bd240d6-kube-api-access-5zpmj\") pod \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\" (UID: \"296dc6e5-d09d-46ae-ade5-8f448bd240d6\") " Jul 05 13:48:44.058291 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.058098 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7tv4\" (UniqueName: \"kubernetes.io/projected/2eec340b-e114-4fdd-b435-7e118b909439-kube-api-access-r7tv4\") pod \"2eec340b-e114-4fdd-b435-7e118b909439\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " Jul 05 13:48:44.058291 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.058117 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-bundle\") pod \"2eec340b-e114-4fdd-b435-7e118b909439\" (UID: \"2eec340b-e114-4fdd-b435-7e118b909439\") " Jul 05 13:48:44.058828 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.058795 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-bundle" (OuterVolumeSpecName: "bundle") pod "2eec340b-e114-4fdd-b435-7e118b909439" (UID: "2eec340b-e114-4fdd-b435-7e118b909439"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:44.059638 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.059596 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-bundle" (OuterVolumeSpecName: "bundle") pod "296dc6e5-d09d-46ae-ade5-8f448bd240d6" (UID: "296dc6e5-d09d-46ae-ade5-8f448bd240d6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:44.060658 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.060632 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eec340b-e114-4fdd-b435-7e118b909439-kube-api-access-r7tv4" (OuterVolumeSpecName: "kube-api-access-r7tv4") pod "2eec340b-e114-4fdd-b435-7e118b909439" (UID: "2eec340b-e114-4fdd-b435-7e118b909439"). InnerVolumeSpecName "kube-api-access-r7tv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:48:44.061191 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.061164 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296dc6e5-d09d-46ae-ade5-8f448bd240d6-kube-api-access-5zpmj" (OuterVolumeSpecName: "kube-api-access-5zpmj") pod "296dc6e5-d09d-46ae-ade5-8f448bd240d6" (UID: "296dc6e5-d09d-46ae-ade5-8f448bd240d6"). InnerVolumeSpecName "kube-api-access-5zpmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:48:44.065905 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.065880 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-util" (OuterVolumeSpecName: "util") pod "296dc6e5-d09d-46ae-ade5-8f448bd240d6" (UID: "296dc6e5-d09d-46ae-ade5-8f448bd240d6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:44.066197 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.066181 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-util" (OuterVolumeSpecName: "util") pod "2eec340b-e114-4fdd-b435-7e118b909439" (UID: "2eec340b-e114-4fdd-b435-7e118b909439"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:44.159131 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.159112 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:44.159131 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.159131 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/296dc6e5-d09d-46ae-ade5-8f448bd240d6-util\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:44.159278 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.159140 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zpmj\" (UniqueName: \"kubernetes.io/projected/296dc6e5-d09d-46ae-ade5-8f448bd240d6-kube-api-access-5zpmj\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:44.159278 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.159149 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7tv4\" (UniqueName: \"kubernetes.io/projected/2eec340b-e114-4fdd-b435-7e118b909439-kube-api-access-r7tv4\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:44.159278 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.159160 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:44.159278 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.159167 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2eec340b-e114-4fdd-b435-7e118b909439-util\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:44.791127 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.791100 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" Jul 05 13:48:44.791543 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.791098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h" event={"ID":"2eec340b-e114-4fdd-b435-7e118b909439","Type":"ContainerDied","Data":"99f7ea2d19ecf1a9cea7a12da87b6791975b0c4a43985c0d9e2e0f6346278a72"} Jul 05 13:48:44.791543 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.791216 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99f7ea2d19ecf1a9cea7a12da87b6791975b0c4a43985c0d9e2e0f6346278a72" Jul 05 13:48:44.793063 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.793047 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" Jul 05 13:48:44.793188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.793080 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st" event={"ID":"296dc6e5-d09d-46ae-ade5-8f448bd240d6","Type":"ContainerDied","Data":"1b6a8f5ad8bdae165e30c319447c313f1b50afabaa2e62aac81413a9d95ceeba"} Jul 05 13:48:44.793188 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.793114 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b6a8f5ad8bdae165e30c319447c313f1b50afabaa2e62aac81413a9d95ceeba" Jul 05 13:48:44.937054 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.937031 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:44.944803 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:44.944783 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:45.067860 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.067776 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-bundle\") pod \"28c22f75-7e29-42ea-8a78-032e925944fe\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " Jul 05 13:48:45.067860 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.067824 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-bundle\") pod \"b883cab1-a89b-435e-b330-68119697e3d1\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " Jul 05 13:48:45.068086 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.067866 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-util\") pod \"b883cab1-a89b-435e-b330-68119697e3d1\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " Jul 05 13:48:45.068086 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.067930 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-util\") pod \"28c22f75-7e29-42ea-8a78-032e925944fe\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " Jul 05 13:48:45.068086 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.067965 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkpsn\" (UniqueName: \"kubernetes.io/projected/b883cab1-a89b-435e-b330-68119697e3d1-kube-api-access-kkpsn\") pod \"b883cab1-a89b-435e-b330-68119697e3d1\" (UID: \"b883cab1-a89b-435e-b330-68119697e3d1\") " Jul 05 13:48:45.068086 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.067987 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhxr2\" (UniqueName: \"kubernetes.io/projected/28c22f75-7e29-42ea-8a78-032e925944fe-kube-api-access-fhxr2\") pod \"28c22f75-7e29-42ea-8a78-032e925944fe\" (UID: \"28c22f75-7e29-42ea-8a78-032e925944fe\") " Jul 05 13:48:45.068411 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.068383 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-bundle" (OuterVolumeSpecName: "bundle") pod "28c22f75-7e29-42ea-8a78-032e925944fe" (UID: "28c22f75-7e29-42ea-8a78-032e925944fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:45.068679 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.068623 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-bundle" (OuterVolumeSpecName: "bundle") pod "b883cab1-a89b-435e-b330-68119697e3d1" (UID: "b883cab1-a89b-435e-b330-68119697e3d1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:45.070174 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.070154 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b883cab1-a89b-435e-b330-68119697e3d1-kube-api-access-kkpsn" (OuterVolumeSpecName: "kube-api-access-kkpsn") pod "b883cab1-a89b-435e-b330-68119697e3d1" (UID: "b883cab1-a89b-435e-b330-68119697e3d1"). InnerVolumeSpecName "kube-api-access-kkpsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:48:45.070494 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.070475 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c22f75-7e29-42ea-8a78-032e925944fe-kube-api-access-fhxr2" (OuterVolumeSpecName: "kube-api-access-fhxr2") pod "28c22f75-7e29-42ea-8a78-032e925944fe" (UID: "28c22f75-7e29-42ea-8a78-032e925944fe"). InnerVolumeSpecName "kube-api-access-fhxr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:48:45.075117 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.075038 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-util" (OuterVolumeSpecName: "util") pod "b883cab1-a89b-435e-b330-68119697e3d1" (UID: "b883cab1-a89b-435e-b330-68119697e3d1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:45.076255 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.076214 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-util" (OuterVolumeSpecName: "util") pod "28c22f75-7e29-42ea-8a78-032e925944fe" (UID: "28c22f75-7e29-42ea-8a78-032e925944fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:45.107787 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.107764 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:48:45.108270 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.108250 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:48:45.169012 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.168990 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-util\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:45.169012 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.169010 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkpsn\" (UniqueName: \"kubernetes.io/projected/b883cab1-a89b-435e-b330-68119697e3d1-kube-api-access-kkpsn\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:45.169136 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.169020 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhxr2\" (UniqueName: \"kubernetes.io/projected/28c22f75-7e29-42ea-8a78-032e925944fe-kube-api-access-fhxr2\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:45.169136 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.169029 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c22f75-7e29-42ea-8a78-032e925944fe-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:45.169136 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.169038 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:45.169136 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.169045 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b883cab1-a89b-435e-b330-68119697e3d1-util\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:48:45.799700 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.799617 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" Jul 05 13:48:45.831048 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.799621 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv" event={"ID":"b883cab1-a89b-435e-b330-68119697e3d1","Type":"ContainerDied","Data":"7d5eb3eb0d664d6f0adebd0b5e08d6c9fd6e5030b568e9da32f785b2cfa9bd57"} Jul 05 13:48:45.831048 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.799752 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d5eb3eb0d664d6f0adebd0b5e08d6c9fd6e5030b568e9da32f785b2cfa9bd57" Jul 05 13:48:45.831048 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.801616 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" event={"ID":"28c22f75-7e29-42ea-8a78-032e925944fe","Type":"ContainerDied","Data":"c668fc0c5328b12ad28f5d5586db75cf480282d7ee692976c3bde6f757f09fac"} Jul 05 13:48:45.831048 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.801655 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c668fc0c5328b12ad28f5d5586db75cf480282d7ee692976c3bde6f757f09fac" Jul 05 13:48:45.831048 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:45.801705 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn" Jul 05 13:48:56.913910 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:48:56.913875 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:00.730260 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:00.730230 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:00.830290 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:00.830255 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:01.709405 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:01.709370 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:04.720906 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:04.720874 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:05.314826 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:05.314799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:05.575074 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:05.575045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:05.683475 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:05.683445 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:07.431525 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:07.431446 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:08.527389 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:08.527353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:08.627233 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:08.627201 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:10.430544 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:10.430507 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:11.356153 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:11.356123 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6496d678d9-skfz2"] Jul 05 13:49:11.534027 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:11.533994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:11.658287 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:11.658204 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:13.733566 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:13.733540 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:13.834009 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:13.833978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:17.142416 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:17.142383 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:17.247270 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:17.247237 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:20.452286 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:20.452254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:20.562806 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:20.562771 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:21.774992 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:21.774963 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:21.874785 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:21.874741 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:22.514456 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:22.514414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:22.619127 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:22.619097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:23.136945 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:23.136918 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:23.237208 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:23.237178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:25.404265 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:25.404239 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:25.501775 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:25.501749 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:32.315573 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:32.315541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:32.470486 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:32.470454 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:33.413795 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:33.413763 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:35.585390 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:35.585354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:35.683983 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:35.683953 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:36.375879 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.375824 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6496d678d9-skfz2" podUID="ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" containerName="console" containerID="cri-o://5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b" gracePeriod=15 Jul 05 13:49:36.630193 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.630144 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:49:36.759335 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759295 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m8kp\" (UniqueName: \"kubernetes.io/projected/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-kube-api-access-8m8kp\") pod \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " Jul 05 13:49:36.759457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759350 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-config\") pod \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " Jul 05 13:49:36.759457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759383 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-service-ca\") pod \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " Jul 05 13:49:36.759457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759419 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-serving-cert\") pod \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " Jul 05 13:49:36.759457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759447 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-oauth-serving-cert\") pod \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " Jul 05 13:49:36.759661 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759484 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-trusted-ca-bundle\") pod \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " Jul 05 13:49:36.759661 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759520 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-oauth-config\") pod \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\" (UID: \"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da\") " Jul 05 13:49:36.760054 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759794 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-service-ca" (OuterVolumeSpecName: "service-ca") pod "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" (UID: "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:49:36.760054 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.759841 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-config" (OuterVolumeSpecName: "console-config") pod "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" (UID: "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:49:36.760054 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.760028 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" (UID: "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:49:36.760203 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.760032 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" (UID: "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:49:36.761479 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.761451 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" (UID: "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:49:36.761618 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.761598 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-kube-api-access-8m8kp" (OuterVolumeSpecName: "kube-api-access-8m8kp") pod "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" (UID: "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da"). InnerVolumeSpecName "kube-api-access-8m8kp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:49:36.761668 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.761598 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" (UID: "ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:49:36.860066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.860042 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:49:36.860066 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.860063 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-oauth-serving-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:49:36.860178 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.860072 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-trusted-ca-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:49:36.860178 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.860080 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-oauth-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:49:36.860178 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.860088 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m8kp\" (UniqueName: \"kubernetes.io/projected/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-kube-api-access-8m8kp\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:49:36.860178 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.860098 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-console-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:49:36.860178 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:36.860106 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:49:37.038103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.038026 2560 generic.go:356] "Generic (PLEG): container finished" podID="ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" containerID="5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b" exitCode=0 Jul 05 13:49:37.038103 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.038073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6496d678d9-skfz2" event={"ID":"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da","Type":"ContainerDied","Data":"5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b"} Jul 05 13:49:37.038279 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.038128 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6496d678d9-skfz2" event={"ID":"ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da","Type":"ContainerDied","Data":"9f8174cf6303f559af4d8aac27bd62334ec13ffbb0416138c2da03b886826784"} Jul 05 13:49:37.038279 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.038075 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6496d678d9-skfz2" Jul 05 13:49:37.038279 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.038146 2560 scope.go:117] "RemoveContainer" containerID="5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b" Jul 05 13:49:37.047015 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.047002 2560 scope.go:117] "RemoveContainer" containerID="5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b" Jul 05 13:49:37.047254 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:49:37.047239 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b\": container with ID starting with 5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b not found: ID does not exist" containerID="5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b" Jul 05 13:49:37.047292 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.047262 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b"} err="failed to get container status \"5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b\": rpc error: code = NotFound desc = could not find container \"5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b\": container with ID starting with 5940bf014f121fd0684af2d15912f49ddb763b92cd71ffe5602d595b8813d53b not found: ID does not exist" Jul 05 13:49:37.064324 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.064290 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6496d678d9-skfz2"] Jul 05 13:49:37.074836 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.074816 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6496d678d9-skfz2"] Jul 05 13:49:37.590742 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:37.590709 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" path="/var/lib/kubelet/pods/ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da/volumes" Jul 05 13:49:39.915857 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:39.915831 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:40.018691 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:40.018657 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:43.492662 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:43.492631 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:43.590216 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:43.590189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:49:46.278348 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:49:46.278283 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:02.164986 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:02.164958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:02.824200 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:02.824160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:04.627023 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.626985 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:04.627442 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627420 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x68hh"] Jul 05 13:50:04.627697 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627685 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eec340b-e114-4fdd-b435-7e118b909439" containerName="pull" Jul 05 13:50:04.627747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627698 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eec340b-e114-4fdd-b435-7e118b909439" containerName="pull" Jul 05 13:50:04.627747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627707 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" containerName="console" Jul 05 13:50:04.627747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627713 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" containerName="console" Jul 05 13:50:04.627747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627730 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerName="util" Jul 05 13:50:04.627747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627736 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerName="util" Jul 05 13:50:04.627747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627741 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b883cab1-a89b-435e-b330-68119697e3d1" containerName="pull" Jul 05 13:50:04.627747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627749 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b883cab1-a89b-435e-b330-68119697e3d1" containerName="pull" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627757 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eec340b-e114-4fdd-b435-7e118b909439" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627762 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eec340b-e114-4fdd-b435-7e118b909439" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627769 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eec340b-e114-4fdd-b435-7e118b909439" containerName="util" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627774 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eec340b-e114-4fdd-b435-7e118b909439" containerName="util" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627779 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b883cab1-a89b-435e-b330-68119697e3d1" containerName="util" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627784 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b883cab1-a89b-435e-b330-68119697e3d1" containerName="util" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627790 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c22f75-7e29-42ea-8a78-032e925944fe" containerName="pull" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627794 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c22f75-7e29-42ea-8a78-032e925944fe" containerName="pull" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627800 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerName="pull" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627805 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerName="pull" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627810 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b883cab1-a89b-435e-b330-68119697e3d1" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627814 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b883cab1-a89b-435e-b330-68119697e3d1" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627820 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c22f75-7e29-42ea-8a78-032e925944fe" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627824 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c22f75-7e29-42ea-8a78-032e925944fe" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627831 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c22f75-7e29-42ea-8a78-032e925944fe" containerName="util" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627836 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c22f75-7e29-42ea-8a78-032e925944fe" containerName="util" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627841 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627845 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627905 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eec340b-e114-4fdd-b435-7e118b909439" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627912 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b883cab1-a89b-435e-b330-68119697e3d1" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627918 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c22f75-7e29-42ea-8a78-032e925944fe" containerName="extract" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627924 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca51ef2e-a8e3-4fdb-9402-b5a3c401e5da" containerName="console" Jul 05 13:50:04.627960 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.627931 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="296dc6e5-d09d-46ae-ade5-8f448bd240d6" containerName="extract" Jul 05 13:50:04.631818 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.631800 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:04.635611 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.635587 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 05 13:50:04.635754 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.635670 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 13:50:04.635754 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.635677 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 13:50:04.636610 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.636594 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ljxll\"" Jul 05 13:50:04.653628 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.653607 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x68hh"] Jul 05 13:50:04.706405 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.706373 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x68hh"] Jul 05 13:50:04.720223 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.720192 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:04.758374 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.758348 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/240b812a-b81e-42e5-8a1c-04114f5c2977-config-file\") pod \"limitador-limitador-64c94fd64f-x68hh\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:04.758488 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.758383 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4tv9\" (UniqueName: \"kubernetes.io/projected/240b812a-b81e-42e5-8a1c-04114f5c2977-kube-api-access-n4tv9\") pod \"limitador-limitador-64c94fd64f-x68hh\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:04.793436 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.793412 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:04.858786 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.858749 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/240b812a-b81e-42e5-8a1c-04114f5c2977-config-file\") pod \"limitador-limitador-64c94fd64f-x68hh\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:04.858956 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.858789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4tv9\" (UniqueName: \"kubernetes.io/projected/240b812a-b81e-42e5-8a1c-04114f5c2977-kube-api-access-n4tv9\") pod \"limitador-limitador-64c94fd64f-x68hh\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:04.859468 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.859441 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/240b812a-b81e-42e5-8a1c-04114f5c2977-config-file\") pod \"limitador-limitador-64c94fd64f-x68hh\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:04.867956 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.867932 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4tv9\" (UniqueName: \"kubernetes.io/projected/240b812a-b81e-42e5-8a1c-04114f5c2977-kube-api-access-n4tv9\") pod \"limitador-limitador-64c94fd64f-x68hh\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:04.941522 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:04.941435 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:05.078522 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:05.078500 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x68hh"] Jul 05 13:50:05.080923 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:50:05.080896 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod240b812a_b81e_42e5_8a1c_04114f5c2977.slice/crio-17df7604cdc778169027b3409bc531c7cdf17b15a6ac85f1059ec0928c05e558 WatchSource:0}: Error finding container 17df7604cdc778169027b3409bc531c7cdf17b15a6ac85f1059ec0928c05e558: Status 404 returned error can't find the container with id 17df7604cdc778169027b3409bc531c7cdf17b15a6ac85f1059ec0928c05e558 Jul 05 13:50:05.082817 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:05.082802 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:50:05.165927 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:05.165902 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" event={"ID":"240b812a-b81e-42e5-8a1c-04114f5c2977","Type":"ContainerStarted","Data":"17df7604cdc778169027b3409bc531c7cdf17b15a6ac85f1059ec0928c05e558"} Jul 05 13:50:05.507082 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:05.507052 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:05.607611 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:05.607584 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:08.183536 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:08.183450 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" event={"ID":"240b812a-b81e-42e5-8a1c-04114f5c2977","Type":"ContainerStarted","Data":"10795449ed6720dc0cd577bc30881ec47f43cc23799352fb6e4e04a566d18686"} Jul 05 13:50:08.183893 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:08.183637 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:08.212702 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:08.212658 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" podStartSLOduration=1.4575852120000001 podStartE2EDuration="4.212644858s" totalImagesPullingTime="2.755059646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:50:08.212450714 +0000 UTC m=+684.392293160" watchObservedRunningTime="2026-07-05 13:50:08.212644858 +0000 UTC m=+684.392487303" Jul 05 13:50:19.188877 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:19.188848 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:19.858774 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:19.858744 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x68hh"] Jul 05 13:50:19.858952 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:19.858915 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" podUID="240b812a-b81e-42e5-8a1c-04114f5c2977" containerName="limitador" containerID="cri-o://10795449ed6720dc0cd577bc30881ec47f43cc23799352fb6e4e04a566d18686" gracePeriod=30 Jul 05 13:50:20.238906 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:20.238874 2560 generic.go:356] "Generic (PLEG): container finished" podID="240b812a-b81e-42e5-8a1c-04114f5c2977" containerID="10795449ed6720dc0cd577bc30881ec47f43cc23799352fb6e4e04a566d18686" exitCode=0 Jul 05 13:50:20.239196 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:20.238944 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" event={"ID":"240b812a-b81e-42e5-8a1c-04114f5c2977","Type":"ContainerDied","Data":"10795449ed6720dc0cd577bc30881ec47f43cc23799352fb6e4e04a566d18686"} Jul 05 13:50:20.806506 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:20.806484 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:20.981868 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:20.981803 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4tv9\" (UniqueName: \"kubernetes.io/projected/240b812a-b81e-42e5-8a1c-04114f5c2977-kube-api-access-n4tv9\") pod \"240b812a-b81e-42e5-8a1c-04114f5c2977\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " Jul 05 13:50:20.981868 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:20.981851 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/240b812a-b81e-42e5-8a1c-04114f5c2977-config-file\") pod \"240b812a-b81e-42e5-8a1c-04114f5c2977\" (UID: \"240b812a-b81e-42e5-8a1c-04114f5c2977\") " Jul 05 13:50:20.982222 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:20.982198 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/240b812a-b81e-42e5-8a1c-04114f5c2977-config-file" (OuterVolumeSpecName: "config-file") pod "240b812a-b81e-42e5-8a1c-04114f5c2977" (UID: "240b812a-b81e-42e5-8a1c-04114f5c2977"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:50:20.983721 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:20.983691 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/240b812a-b81e-42e5-8a1c-04114f5c2977-kube-api-access-n4tv9" (OuterVolumeSpecName: "kube-api-access-n4tv9") pod "240b812a-b81e-42e5-8a1c-04114f5c2977" (UID: "240b812a-b81e-42e5-8a1c-04114f5c2977"). InnerVolumeSpecName "kube-api-access-n4tv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:50:21.082747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.082724 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4tv9\" (UniqueName: \"kubernetes.io/projected/240b812a-b81e-42e5-8a1c-04114f5c2977-kube-api-access-n4tv9\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:50:21.082844 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.082751 2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/240b812a-b81e-42e5-8a1c-04114f5c2977-config-file\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:50:21.244500 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.244430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" event={"ID":"240b812a-b81e-42e5-8a1c-04114f5c2977","Type":"ContainerDied","Data":"17df7604cdc778169027b3409bc531c7cdf17b15a6ac85f1059ec0928c05e558"} Jul 05 13:50:21.244500 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.244451 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x68hh" Jul 05 13:50:21.244500 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.244487 2560 scope.go:117] "RemoveContainer" containerID="10795449ed6720dc0cd577bc30881ec47f43cc23799352fb6e4e04a566d18686" Jul 05 13:50:21.269148 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.269123 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x68hh"] Jul 05 13:50:21.274967 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.274946 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x68hh"] Jul 05 13:50:21.589755 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:21.589725 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="240b812a-b81e-42e5-8a1c-04114f5c2977" path="/var/lib/kubelet/pods/240b812a-b81e-42e5-8a1c-04114f5c2977/volumes" Jul 05 13:50:26.144064 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.144034 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-q2dt7"] Jul 05 13:50:26.144430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.144302 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="240b812a-b81e-42e5-8a1c-04114f5c2977" containerName="limitador" Jul 05 13:50:26.144430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.144330 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="240b812a-b81e-42e5-8a1c-04114f5c2977" containerName="limitador" Jul 05 13:50:26.144430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.144381 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="240b812a-b81e-42e5-8a1c-04114f5c2977" containerName="limitador" Jul 05 13:50:26.148538 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.148519 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.153287 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.153259 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 05 13:50:26.153979 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.153274 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5vksw\"" Jul 05 13:50:26.159101 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.159072 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-q2dt7"] Jul 05 13:50:26.179383 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.179357 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:26.320438 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.320411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/daa74441-440d-4e47-92b7-360c72f653ff-data\") pod \"postgres-69fb4594cf-q2dt7\" (UID: \"daa74441-440d-4e47-92b7-360c72f653ff\") " pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.320629 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.320467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl2cq\" (UniqueName: \"kubernetes.io/projected/daa74441-440d-4e47-92b7-360c72f653ff-kube-api-access-jl2cq\") pod \"postgres-69fb4594cf-q2dt7\" (UID: \"daa74441-440d-4e47-92b7-360c72f653ff\") " pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.420822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.420753 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/daa74441-440d-4e47-92b7-360c72f653ff-data\") pod \"postgres-69fb4594cf-q2dt7\" (UID: \"daa74441-440d-4e47-92b7-360c72f653ff\") " pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.420822 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.420803 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl2cq\" (UniqueName: \"kubernetes.io/projected/daa74441-440d-4e47-92b7-360c72f653ff-kube-api-access-jl2cq\") pod \"postgres-69fb4594cf-q2dt7\" (UID: \"daa74441-440d-4e47-92b7-360c72f653ff\") " pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.421171 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.421149 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/daa74441-440d-4e47-92b7-360c72f653ff-data\") pod \"postgres-69fb4594cf-q2dt7\" (UID: \"daa74441-440d-4e47-92b7-360c72f653ff\") " pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.431505 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.431479 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl2cq\" (UniqueName: \"kubernetes.io/projected/daa74441-440d-4e47-92b7-360c72f653ff-kube-api-access-jl2cq\") pod \"postgres-69fb4594cf-q2dt7\" (UID: \"daa74441-440d-4e47-92b7-360c72f653ff\") " pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.459672 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.459650 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:26.593406 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:26.593381 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-q2dt7"] Jul 05 13:50:26.594956 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:50:26.594931 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa74441_440d_4e47_92b7_360c72f653ff.slice/crio-8c00504c3b2c8d7a66cbf21b8ceee9d2029e8f0b41b16078e2b1877f472bb12a WatchSource:0}: Error finding container 8c00504c3b2c8d7a66cbf21b8ceee9d2029e8f0b41b16078e2b1877f472bb12a: Status 404 returned error can't find the container with id 8c00504c3b2c8d7a66cbf21b8ceee9d2029e8f0b41b16078e2b1877f472bb12a Jul 05 13:50:27.277785 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:27.277700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-q2dt7" event={"ID":"daa74441-440d-4e47-92b7-360c72f653ff","Type":"ContainerStarted","Data":"8c00504c3b2c8d7a66cbf21b8ceee9d2029e8f0b41b16078e2b1877f472bb12a"} Jul 05 13:50:31.303765 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:31.303733 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-q2dt7" event={"ID":"daa74441-440d-4e47-92b7-360c72f653ff","Type":"ContainerStarted","Data":"69744589d782b09ccd0ccd99e1397aca1b505bc2700fe0437a05b7d1035b290f"} Jul 05 13:50:31.304156 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:31.303864 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:31.326011 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:31.325964 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-q2dt7" podStartSLOduration=0.808846582 podStartE2EDuration="5.325950807s" totalImagesPullingTime="4.517104225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:50:31.324561618 +0000 UTC m=+707.504404059" watchObservedRunningTime="2026-07-05 13:50:31.325950807 +0000 UTC m=+707.505793268" Jul 05 13:50:37.337496 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:37.337423 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-q2dt7" Jul 05 13:50:37.677423 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:37.677340 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:37.884326 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:37.884279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:40.686641 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:40.686608 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:40.786324 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:40.786280 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:44.083448 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.083417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:44.094555 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.094525 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f8cf45755-rg9bn"] Jul 05 13:50:44.105345 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.105323 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.108691 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.108669 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 13:50:44.108823 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.108717 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6wtqp\"" Jul 05 13:50:44.108823 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.108776 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 05 13:50:44.114360 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.114338 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8cf45755-rg9bn"] Jul 05 13:50:44.148354 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.148305 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x27qk\" (UniqueName: \"kubernetes.io/projected/d09d158b-99fe-4d6e-a4e8-aceee98528c9-kube-api-access-x27qk\") pod \"maas-controller-7f8cf45755-rg9bn\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.148474 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.148403 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d158b-99fe-4d6e-a4e8-aceee98528c9-webhook-cert\") pod \"maas-controller-7f8cf45755-rg9bn\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.185605 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.185578 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:44.249529 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.249502 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x27qk\" (UniqueName: \"kubernetes.io/projected/d09d158b-99fe-4d6e-a4e8-aceee98528c9-kube-api-access-x27qk\") pod \"maas-controller-7f8cf45755-rg9bn\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.249637 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.249534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d158b-99fe-4d6e-a4e8-aceee98528c9-webhook-cert\") pod \"maas-controller-7f8cf45755-rg9bn\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.251995 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.251974 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d158b-99fe-4d6e-a4e8-aceee98528c9-webhook-cert\") pod \"maas-controller-7f8cf45755-rg9bn\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.261969 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.261945 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x27qk\" (UniqueName: \"kubernetes.io/projected/d09d158b-99fe-4d6e-a4e8-aceee98528c9-kube-api-access-x27qk\") pod \"maas-controller-7f8cf45755-rg9bn\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.268519 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.268496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:44.273084 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.273064 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-696f68cd7-6q46k"] Jul 05 13:50:44.275249 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.275232 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.289387 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.289367 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-696f68cd7-6q46k"] Jul 05 13:50:44.350706 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.350655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94425c89-fa4f-4a5a-80ae-c0375778b6de-webhook-cert\") pod \"maas-controller-696f68cd7-6q46k\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.350706 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.350684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbms4\" (UniqueName: \"kubernetes.io/projected/94425c89-fa4f-4a5a-80ae-c0375778b6de-kube-api-access-sbms4\") pod \"maas-controller-696f68cd7-6q46k\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.418119 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.418097 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:44.451274 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.451249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94425c89-fa4f-4a5a-80ae-c0375778b6de-webhook-cert\") pod \"maas-controller-696f68cd7-6q46k\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.451391 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.451283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbms4\" (UniqueName: \"kubernetes.io/projected/94425c89-fa4f-4a5a-80ae-c0375778b6de-kube-api-access-sbms4\") pod \"maas-controller-696f68cd7-6q46k\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.453588 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.453569 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94425c89-fa4f-4a5a-80ae-c0375778b6de-webhook-cert\") pod \"maas-controller-696f68cd7-6q46k\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.460582 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.460563 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbms4\" (UniqueName: \"kubernetes.io/projected/94425c89-fa4f-4a5a-80ae-c0375778b6de-kube-api-access-sbms4\") pod \"maas-controller-696f68cd7-6q46k\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.543454 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.543430 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f8cf45755-rg9bn"] Jul 05 13:50:44.545367 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:50:44.545344 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09d158b_99fe_4d6e_a4e8_aceee98528c9.slice/crio-71a6438d6d5ebf4ab85c55a02bbf420fe4b5e8e3d30ba876c7b2c9191d57cb98 WatchSource:0}: Error finding container 71a6438d6d5ebf4ab85c55a02bbf420fe4b5e8e3d30ba876c7b2c9191d57cb98: Status 404 returned error can't find the container with id 71a6438d6d5ebf4ab85c55a02bbf420fe4b5e8e3d30ba876c7b2c9191d57cb98 Jul 05 13:50:44.584739 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.584720 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:44.711325 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:44.711272 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-696f68cd7-6q46k"] Jul 05 13:50:44.716516 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:50:44.716491 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94425c89_fa4f_4a5a_80ae_c0375778b6de.slice/crio-e3778b8951415e91de23645ae4c0adb13ea0397dcecc1659c8926c5a2fd102e1 WatchSource:0}: Error finding container e3778b8951415e91de23645ae4c0adb13ea0397dcecc1659c8926c5a2fd102e1: Status 404 returned error can't find the container with id e3778b8951415e91de23645ae4c0adb13ea0397dcecc1659c8926c5a2fd102e1 Jul 05 13:50:45.373343 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:45.373238 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" event={"ID":"d09d158b-99fe-4d6e-a4e8-aceee98528c9","Type":"ContainerStarted","Data":"71a6438d6d5ebf4ab85c55a02bbf420fe4b5e8e3d30ba876c7b2c9191d57cb98"} Jul 05 13:50:45.375194 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:45.375122 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-696f68cd7-6q46k" event={"ID":"94425c89-fa4f-4a5a-80ae-c0375778b6de","Type":"ContainerStarted","Data":"e3778b8951415e91de23645ae4c0adb13ea0397dcecc1659c8926c5a2fd102e1"} Jul 05 13:50:46.528533 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:46.528506 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 13:50:46.751182 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:46.751144 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:46.838798 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:46.838767 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:50:47.386583 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:47.386550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" event={"ID":"d09d158b-99fe-4d6e-a4e8-aceee98528c9","Type":"ContainerStarted","Data":"29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f"} Jul 05 13:50:47.386764 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:47.386662 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:47.388102 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:47.388070 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-696f68cd7-6q46k" event={"ID":"94425c89-fa4f-4a5a-80ae-c0375778b6de","Type":"ContainerStarted","Data":"d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f"} Jul 05 13:50:47.388221 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:47.388119 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:47.438090 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:47.438041 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" podStartSLOduration=1.458928207 podStartE2EDuration="3.438025395s" totalImagesPullingTime="1.979097188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:50:47.434820215 +0000 UTC m=+723.614662674" watchObservedRunningTime="2026-07-05 13:50:47.438025395 +0000 UTC m=+723.617867841" Jul 05 13:50:47.467576 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:47.467522 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-696f68cd7-6q46k" podStartSLOduration=1.65257708 podStartE2EDuration="3.46750593s" totalImagesPullingTime="1.81492885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:50:47.467039924 +0000 UTC m=+723.646882370" watchObservedRunningTime="2026-07-05 13:50:47.46750593 +0000 UTC m=+723.647348377" Jul 05 13:50:58.404207 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.404171 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:50:58.404778 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.404261 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:58.472097 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.472068 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f8cf45755-rg9bn"] Jul 05 13:50:58.472259 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.472231 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" podUID="d09d158b-99fe-4d6e-a4e8-aceee98528c9" containerName="manager" containerID="cri-o://29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f" gracePeriod=30 Jul 05 13:50:58.711572 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.711548 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:58.764730 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.764706 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d158b-99fe-4d6e-a4e8-aceee98528c9-webhook-cert\") pod \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " Jul 05 13:50:58.764899 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.764765 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x27qk\" (UniqueName: \"kubernetes.io/projected/d09d158b-99fe-4d6e-a4e8-aceee98528c9-kube-api-access-x27qk\") pod \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\" (UID: \"d09d158b-99fe-4d6e-a4e8-aceee98528c9\") " Jul 05 13:50:58.766828 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.766803 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d09d158b-99fe-4d6e-a4e8-aceee98528c9-kube-api-access-x27qk" (OuterVolumeSpecName: "kube-api-access-x27qk") pod "d09d158b-99fe-4d6e-a4e8-aceee98528c9" (UID: "d09d158b-99fe-4d6e-a4e8-aceee98528c9"). InnerVolumeSpecName "kube-api-access-x27qk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:50:58.766945 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.766850 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09d158b-99fe-4d6e-a4e8-aceee98528c9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d09d158b-99fe-4d6e-a4e8-aceee98528c9" (UID: "d09d158b-99fe-4d6e-a4e8-aceee98528c9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:50:58.866062 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.866037 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d158b-99fe-4d6e-a4e8-aceee98528c9-webhook-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:50:58.866062 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:58.866061 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x27qk\" (UniqueName: \"kubernetes.io/projected/d09d158b-99fe-4d6e-a4e8-aceee98528c9-kube-api-access-x27qk\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:50:59.449566 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.449526 2560 generic.go:356] "Generic (PLEG): container finished" podID="d09d158b-99fe-4d6e-a4e8-aceee98528c9" containerID="29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f" exitCode=0 Jul 05 13:50:59.449943 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.449580 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" Jul 05 13:50:59.449943 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.449618 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" event={"ID":"d09d158b-99fe-4d6e-a4e8-aceee98528c9","Type":"ContainerDied","Data":"29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f"} Jul 05 13:50:59.449943 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.449696 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f8cf45755-rg9bn" event={"ID":"d09d158b-99fe-4d6e-a4e8-aceee98528c9","Type":"ContainerDied","Data":"71a6438d6d5ebf4ab85c55a02bbf420fe4b5e8e3d30ba876c7b2c9191d57cb98"} Jul 05 13:50:59.449943 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.449719 2560 scope.go:117] "RemoveContainer" containerID="29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f" Jul 05 13:50:59.458842 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.458826 2560 scope.go:117] "RemoveContainer" containerID="29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f" Jul 05 13:50:59.459075 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:50:59.459056 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f\": container with ID starting with 29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f not found: ID does not exist" containerID="29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f" Jul 05 13:50:59.459122 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.459083 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f"} err="failed to get container status \"29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f\": rpc error: code = NotFound desc = could not find container \"29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f\": container with ID starting with 29bfeac690c440868dfc77fe92d9ef8d2e15c9578bc817475f583a94177c0a9f not found: ID does not exist" Jul 05 13:50:59.472878 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.472854 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f8cf45755-rg9bn"] Jul 05 13:50:59.478114 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.478087 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f8cf45755-rg9bn"] Jul 05 13:50:59.590865 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:50:59.590842 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d09d158b-99fe-4d6e-a4e8-aceee98528c9" path="/var/lib/kubelet/pods/d09d158b-99fe-4d6e-a4e8-aceee98528c9/volumes" Jul 05 13:51:00.589068 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:00.589039 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:00.644773 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:00.644746 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:18.294701 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.294659 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:18.787924 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.787898 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:18.888518 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.888483 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5b59cd5649-b68dz"] Jul 05 13:51:18.888778 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.888766 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09d158b-99fe-4d6e-a4e8-aceee98528c9" containerName="manager" Jul 05 13:51:18.888826 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.888780 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09d158b-99fe-4d6e-a4e8-aceee98528c9" containerName="manager" Jul 05 13:51:18.888861 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.888838 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d09d158b-99fe-4d6e-a4e8-aceee98528c9" containerName="manager" Jul 05 13:51:18.892005 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.891989 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:18.895001 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.894975 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 05 13:51:18.895001 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.894993 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-g49nn\"" Jul 05 13:51:18.903706 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:18.903678 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5b59cd5649-b68dz"] Jul 05 13:51:19.021953 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.021921 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj9s6\" (UniqueName: \"kubernetes.io/projected/70a26def-7569-4331-a895-742379fdeeed-kube-api-access-fj9s6\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.021953 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.021953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a26def-7569-4331-a895-742379fdeeed-openshift-service-ca\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.022120 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.022033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a26def-7569-4331-a895-742379fdeeed-maas-api-tls\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.072881 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.072805 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:19.123092 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.123060 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9s6\" (UniqueName: \"kubernetes.io/projected/70a26def-7569-4331-a895-742379fdeeed-kube-api-access-fj9s6\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.123092 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.123093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a26def-7569-4331-a895-742379fdeeed-openshift-service-ca\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.123264 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.123143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a26def-7569-4331-a895-742379fdeeed-maas-api-tls\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.123872 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.123853 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70a26def-7569-4331-a895-742379fdeeed-openshift-service-ca\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.125536 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.125517 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70a26def-7569-4331-a895-742379fdeeed-maas-api-tls\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.143823 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.143799 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9s6\" (UniqueName: \"kubernetes.io/projected/70a26def-7569-4331-a895-742379fdeeed-kube-api-access-fj9s6\") pod \"maas-api-5b59cd5649-b68dz\" (UID: \"70a26def-7569-4331-a895-742379fdeeed\") " pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.201777 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.201745 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:19.342623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.342591 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5b59cd5649-b68dz"] Jul 05 13:51:19.344523 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:51:19.344490 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a26def_7569_4331_a895_742379fdeeed.slice/crio-43cec0576233b9901b1a9938b6ade85d0227feb983c9894fd33b18e4f4985127 WatchSource:0}: Error finding container 43cec0576233b9901b1a9938b6ade85d0227feb983c9894fd33b18e4f4985127: Status 404 returned error can't find the container with id 43cec0576233b9901b1a9938b6ade85d0227feb983c9894fd33b18e4f4985127 Jul 05 13:51:19.350421 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.350383 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:19.452233 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.452205 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:19.485523 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.485490 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-q7r8k"] Jul 05 13:51:19.490069 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.490050 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.493020 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.492992 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vdmwt\"" Jul 05 13:51:19.493124 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.493005 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 05 13:51:19.498749 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.498730 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-q7r8k"] Jul 05 13:51:19.546989 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.546965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5b59cd5649-b68dz" event={"ID":"70a26def-7569-4331-a895-742379fdeeed","Type":"ContainerStarted","Data":"43cec0576233b9901b1a9938b6ade85d0227feb983c9894fd33b18e4f4985127"} Jul 05 13:51:19.628779 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.628716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg2nk\" (UniqueName: \"kubernetes.io/projected/e1c17b93-b079-4979-895f-c4c90fadb951-kube-api-access-tg2nk\") pod \"payload-processing-58f86c7798-q7r8k\" (UID: \"e1c17b93-b079-4979-895f-c4c90fadb951\") " pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.628881 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.628798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c17b93-b079-4979-895f-c4c90fadb951-plugins-config-volume\") pod \"payload-processing-58f86c7798-q7r8k\" (UID: \"e1c17b93-b079-4979-895f-c4c90fadb951\") " pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.730144 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.730114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg2nk\" (UniqueName: \"kubernetes.io/projected/e1c17b93-b079-4979-895f-c4c90fadb951-kube-api-access-tg2nk\") pod \"payload-processing-58f86c7798-q7r8k\" (UID: \"e1c17b93-b079-4979-895f-c4c90fadb951\") " pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.730284 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.730205 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c17b93-b079-4979-895f-c4c90fadb951-plugins-config-volume\") pod \"payload-processing-58f86c7798-q7r8k\" (UID: \"e1c17b93-b079-4979-895f-c4c90fadb951\") " pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.730945 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.730922 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c17b93-b079-4979-895f-c4c90fadb951-plugins-config-volume\") pod \"payload-processing-58f86c7798-q7r8k\" (UID: \"e1c17b93-b079-4979-895f-c4c90fadb951\") " pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.739061 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.739040 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg2nk\" (UniqueName: \"kubernetes.io/projected/e1c17b93-b079-4979-895f-c4c90fadb951-kube-api-access-tg2nk\") pod \"payload-processing-58f86c7798-q7r8k\" (UID: \"e1c17b93-b079-4979-895f-c4c90fadb951\") " pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.801048 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.801025 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:19.926401 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:19.926377 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-q7r8k"] Jul 05 13:51:19.927979 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:51:19.927953 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c17b93_b079_4979_895f_c4c90fadb951.slice/crio-0a7b1f4d13a01318aeccfcd5223679f71c762e6eaf8c9d97605a1d19b47d971d WatchSource:0}: Error finding container 0a7b1f4d13a01318aeccfcd5223679f71c762e6eaf8c9d97605a1d19b47d971d: Status 404 returned error can't find the container with id 0a7b1f4d13a01318aeccfcd5223679f71c762e6eaf8c9d97605a1d19b47d971d Jul 05 13:51:20.553826 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:20.553785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" event={"ID":"e1c17b93-b079-4979-895f-c4c90fadb951","Type":"ContainerStarted","Data":"0a7b1f4d13a01318aeccfcd5223679f71c762e6eaf8c9d97605a1d19b47d971d"} Jul 05 13:51:21.562381 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:21.562346 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5b59cd5649-b68dz" event={"ID":"70a26def-7569-4331-a895-742379fdeeed","Type":"ContainerStarted","Data":"0e4764926e748a31ff94f72b51086f7054eaae6977ffde659bb29927ceb008d2"} Jul 05 13:51:21.562838 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:21.562479 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:21.583999 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:21.583947 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5b59cd5649-b68dz" podStartSLOduration=2.187844323 podStartE2EDuration="3.583929398s" totalImagesPullingTime="1.396085075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:21.580748167 +0000 UTC m=+757.760590616" watchObservedRunningTime="2026-07-05 13:51:21.583929398 +0000 UTC m=+757.763771845" Jul 05 13:51:22.568101 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:22.568061 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" event={"ID":"e1c17b93-b079-4979-895f-c4c90fadb951","Type":"ContainerStarted","Data":"4cc81f40ed931bf2270bea49e72438ab462805f344eb4369b873b0e1e5ee07f2"} Jul 05 13:51:22.588580 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:22.588533 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" podStartSLOduration=1.863137487 podStartE2EDuration="3.5885189s" totalImagesPullingTime="1.725381413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:22.587825199 +0000 UTC m=+758.767667646" watchObservedRunningTime="2026-07-05 13:51:22.5885189 +0000 UTC m=+758.768361350" Jul 05 13:51:23.572469 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:23.572439 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:27.573460 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:27.573425 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5b59cd5649-b68dz" Jul 05 13:51:28.541096 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:28.541062 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:28.630495 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:28.630465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:34.008624 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.008586 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk"] Jul 05 13:51:34.011347 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.011327 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.014264 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.014240 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 13:51:34.015457 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.015435 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 05 13:51:34.015548 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.015443 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tj5dq\"" Jul 05 13:51:34.015548 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.015495 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 13:51:34.022348 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.022325 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk"] Jul 05 13:51:34.026891 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.026868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:34.153539 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.153500 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.153773 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.153551 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.153773 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.153583 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.153773 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.153610 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.153773 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.153644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b93bc5-971e-4b93-9630-bbb9f8193275-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.153773 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.153673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfcj7\" (UniqueName: \"kubernetes.io/projected/61b93bc5-971e-4b93-9630-bbb9f8193275-kube-api-access-sfcj7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.254570 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.254534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.254570 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.254581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b93bc5-971e-4b93-9630-bbb9f8193275-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.254819 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.254603 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfcj7\" (UniqueName: \"kubernetes.io/projected/61b93bc5-971e-4b93-9630-bbb9f8193275-kube-api-access-sfcj7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.254819 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.254651 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.254819 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.254695 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.254819 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.254717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.255037 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.254990 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.255037 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.255023 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.255224 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.255205 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.256978 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.256952 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b93bc5-971e-4b93-9630-bbb9f8193275-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.257235 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.257217 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b93bc5-971e-4b93-9630-bbb9f8193275-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.262768 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.262710 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfcj7\" (UniqueName: \"kubernetes.io/projected/61b93bc5-971e-4b93-9630-bbb9f8193275-kube-api-access-sfcj7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk\" (UID: \"61b93bc5-971e-4b93-9630-bbb9f8193275\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.321696 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.321659 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:34.454670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.454643 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk"] Jul 05 13:51:34.456116 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:51:34.456090 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b93bc5_971e_4b93_9630_bbb9f8193275.slice/crio-c95b5057d0647ac537cc334d2616df2ee224eafb48e8385287d8c219c2214529 WatchSource:0}: Error finding container c95b5057d0647ac537cc334d2616df2ee224eafb48e8385287d8c219c2214529: Status 404 returned error can't find the container with id c95b5057d0647ac537cc334d2616df2ee224eafb48e8385287d8c219c2214529 Jul 05 13:51:34.579273 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.579251 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-q7r8k" Jul 05 13:51:34.629303 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:34.629266 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" event={"ID":"61b93bc5-971e-4b93-9630-bbb9f8193275","Type":"ContainerStarted","Data":"c95b5057d0647ac537cc334d2616df2ee224eafb48e8385287d8c219c2214529"} Jul 05 13:51:35.636514 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:35.636479 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" event={"ID":"61b93bc5-971e-4b93-9630-bbb9f8193275","Type":"ContainerStarted","Data":"20b537c1e63573cbc96f2e2f84f5d518b2b7de8aac6c0af0cd686b3aadfb39a5"} Jul 05 13:51:35.657430 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:35.657384 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" podStartSLOduration=1.5680155070000001 podStartE2EDuration="2.657368245s" totalImagesPullingTime="1.089352738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:35.654996354 +0000 UTC m=+771.834838801" watchObservedRunningTime="2026-07-05 13:51:35.657368245 +0000 UTC m=+771.837210691" Jul 05 13:51:40.515395 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:40.515361 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:44.322418 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:44.322373 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:44.322890 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:44.322446 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:44.334814 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:44.334786 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:44.694577 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:44.694548 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk" Jul 05 13:51:49.377935 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:49.377895 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:51:57.117827 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:51:57.117791 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:01.592888 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.592857 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p"] Jul 05 13:52:01.598028 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.598011 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.601147 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.601130 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 05 13:52:01.605912 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.605887 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p"] Jul 05 13:52:01.628868 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.628846 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:52:01.689697 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.689675 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f365a76e-c6ce-4d57-a86b-a50b64e16382-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.689824 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.689717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5frgw\" (UniqueName: \"kubernetes.io/projected/f365a76e-c6ce-4d57-a86b-a50b64e16382-kube-api-access-5frgw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.689824 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.689755 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.689824 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.689802 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.689954 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.689866 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.689954 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.689892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791319 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791319 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791324 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791551 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791551 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f365a76e-c6ce-4d57-a86b-a50b64e16382-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791551 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5frgw\" (UniqueName: \"kubernetes.io/projected/f365a76e-c6ce-4d57-a86b-a50b64e16382-kube-api-access-5frgw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791747 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791558 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791807 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791782 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791915 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791893 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.791985 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.791916 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.793737 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.793716 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f365a76e-c6ce-4d57-a86b-a50b64e16382-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.793926 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.793909 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f365a76e-c6ce-4d57-a86b-a50b64e16382-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.800544 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.800510 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5frgw\" (UniqueName: \"kubernetes.io/projected/f365a76e-c6ce-4d57-a86b-a50b64e16382-kube-api-access-5frgw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j7s2p\" (UID: \"f365a76e-c6ce-4d57-a86b-a50b64e16382\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:01.908465 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:01.908396 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:02.042439 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:02.042412 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p"] Jul 05 13:52:02.044425 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:52:02.044392 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf365a76e_c6ce_4d57_a86b_a50b64e16382.slice/crio-ffe311ae73025bde798d9efac37a22f2a58365cc9c6a92be67e9e048babb5d03 WatchSource:0}: Error finding container ffe311ae73025bde798d9efac37a22f2a58365cc9c6a92be67e9e048babb5d03: Status 404 returned error can't find the container with id ffe311ae73025bde798d9efac37a22f2a58365cc9c6a92be67e9e048babb5d03 Jul 05 13:52:02.762875 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:02.762837 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" event={"ID":"f365a76e-c6ce-4d57-a86b-a50b64e16382","Type":"ContainerStarted","Data":"441628f95bf703f032a25cd82b0f886e21e0d0dc78d053266f283c4e1e715a71"} Jul 05 13:52:02.763218 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:02.762901 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" event={"ID":"f365a76e-c6ce-4d57-a86b-a50b64e16382","Type":"ContainerStarted","Data":"ffe311ae73025bde798d9efac37a22f2a58365cc9c6a92be67e9e048babb5d03"} Jul 05 13:52:02.782474 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:02.782428 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" podStartSLOduration=1.621706715 podStartE2EDuration="1.782412638s" totalImagesPullingTime="160.705923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:52:02.780631127 +0000 UTC m=+798.960473573" watchObservedRunningTime="2026-07-05 13:52:02.782412638 +0000 UTC m=+798.962255122" Jul 05 13:52:11.908972 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:11.908889 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:11.908972 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:11.908956 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:11.921262 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:11.921240 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:12.827440 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:12.827411 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j7s2p" Jul 05 13:52:17.590839 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:52:17.590803 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:10.634805 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:10.634763 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:10.733841 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:10.733814 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:11.852730 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:11.852701 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:11.941709 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:11.941655 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:13.422107 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:13.422078 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:34.043488 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:34.043439 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:53:45.145231 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:45.145205 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:53:45.146701 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:53:45.146673 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:54:12.557440 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:12.557389 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:54:13.538485 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:13.538443 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:54:33.454551 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:33.454507 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:54:44.775028 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:44.774952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:54:44.886635 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:44.886602 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-696f68cd7-6q46k"] Jul 05 13:54:44.886894 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:44.886864 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-696f68cd7-6q46k" podUID="94425c89-fa4f-4a5a-80ae-c0375778b6de" containerName="manager" containerID="cri-o://d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f" gracePeriod=30 Jul 05 13:54:45.124670 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.124647 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:54:45.268929 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.268898 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbms4\" (UniqueName: \"kubernetes.io/projected/94425c89-fa4f-4a5a-80ae-c0375778b6de-kube-api-access-sbms4\") pod \"94425c89-fa4f-4a5a-80ae-c0375778b6de\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " Jul 05 13:54:45.269114 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.268950 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94425c89-fa4f-4a5a-80ae-c0375778b6de-webhook-cert\") pod \"94425c89-fa4f-4a5a-80ae-c0375778b6de\" (UID: \"94425c89-fa4f-4a5a-80ae-c0375778b6de\") " Jul 05 13:54:45.270940 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.270909 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94425c89-fa4f-4a5a-80ae-c0375778b6de-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "94425c89-fa4f-4a5a-80ae-c0375778b6de" (UID: "94425c89-fa4f-4a5a-80ae-c0375778b6de"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:54:45.270940 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.270908 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94425c89-fa4f-4a5a-80ae-c0375778b6de-kube-api-access-sbms4" (OuterVolumeSpecName: "kube-api-access-sbms4") pod "94425c89-fa4f-4a5a-80ae-c0375778b6de" (UID: "94425c89-fa4f-4a5a-80ae-c0375778b6de"). InnerVolumeSpecName "kube-api-access-sbms4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:54:45.369851 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.369826 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94425c89-fa4f-4a5a-80ae-c0375778b6de-webhook-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:54:45.369851 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.369853 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbms4\" (UniqueName: \"kubernetes.io/projected/94425c89-fa4f-4a5a-80ae-c0375778b6de-kube-api-access-sbms4\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 13:54:45.547664 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.547627 2560 generic.go:356] "Generic (PLEG): container finished" podID="94425c89-fa4f-4a5a-80ae-c0375778b6de" containerID="d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f" exitCode=0 Jul 05 13:54:45.547837 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.547666 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-696f68cd7-6q46k" Jul 05 13:54:45.547837 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.547715 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-696f68cd7-6q46k" event={"ID":"94425c89-fa4f-4a5a-80ae-c0375778b6de","Type":"ContainerDied","Data":"d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f"} Jul 05 13:54:45.547837 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.547770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-696f68cd7-6q46k" event={"ID":"94425c89-fa4f-4a5a-80ae-c0375778b6de","Type":"ContainerDied","Data":"e3778b8951415e91de23645ae4c0adb13ea0397dcecc1659c8926c5a2fd102e1"} Jul 05 13:54:45.547837 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.547795 2560 scope.go:117] "RemoveContainer" containerID="d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f" Jul 05 13:54:45.560676 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.560657 2560 scope.go:117] "RemoveContainer" containerID="d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f" Jul 05 13:54:45.560924 ip-10-0-142-97 kubenswrapper[2560]: E0705 13:54:45.560905 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f\": container with ID starting with d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f not found: ID does not exist" containerID="d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f" Jul 05 13:54:45.560988 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.560936 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f"} err="failed to get container status \"d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f\": rpc error: code = NotFound desc = could not find container \"d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f\": container with ID starting with d0f7dbdf7249a122e06365e0e22893729a01d6baffbe186cce96fd397528c99f not found: ID does not exist" Jul 05 13:54:45.574172 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.574111 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-696f68cd7-6q46k"] Jul 05 13:54:45.576644 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.576617 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-696f68cd7-6q46k"] Jul 05 13:54:45.592197 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:45.592172 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94425c89-fa4f-4a5a-80ae-c0375778b6de" path="/var/lib/kubelet/pods/94425c89-fa4f-4a5a-80ae-c0375778b6de/volumes" Jul 05 13:54:46.234292 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.234252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:54:46.355944 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.355913 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-696f68cd7-v5nnq"] Jul 05 13:54:46.356293 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.356274 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94425c89-fa4f-4a5a-80ae-c0375778b6de" containerName="manager" Jul 05 13:54:46.356375 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.356298 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="94425c89-fa4f-4a5a-80ae-c0375778b6de" containerName="manager" Jul 05 13:54:46.356427 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.356401 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="94425c89-fa4f-4a5a-80ae-c0375778b6de" containerName="manager" Jul 05 13:54:46.360536 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.360517 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.363231 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.363213 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 13:54:46.364410 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.364394 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 05 13:54:46.364468 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.364394 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6wtqp\"" Jul 05 13:54:46.368623 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.368601 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-696f68cd7-v5nnq"] Jul 05 13:54:46.481565 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.481533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51a79347-9247-46d7-b27d-e6826d9c947e-webhook-cert\") pod \"maas-controller-696f68cd7-v5nnq\" (UID: \"51a79347-9247-46d7-b27d-e6826d9c947e\") " pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.481723 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.481579 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5755r\" (UniqueName: \"kubernetes.io/projected/51a79347-9247-46d7-b27d-e6826d9c947e-kube-api-access-5755r\") pod \"maas-controller-696f68cd7-v5nnq\" (UID: \"51a79347-9247-46d7-b27d-e6826d9c947e\") " pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.582574 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.582545 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5755r\" (UniqueName: \"kubernetes.io/projected/51a79347-9247-46d7-b27d-e6826d9c947e-kube-api-access-5755r\") pod \"maas-controller-696f68cd7-v5nnq\" (UID: \"51a79347-9247-46d7-b27d-e6826d9c947e\") " pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.582754 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.582647 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51a79347-9247-46d7-b27d-e6826d9c947e-webhook-cert\") pod \"maas-controller-696f68cd7-v5nnq\" (UID: \"51a79347-9247-46d7-b27d-e6826d9c947e\") " pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.584973 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.584950 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51a79347-9247-46d7-b27d-e6826d9c947e-webhook-cert\") pod \"maas-controller-696f68cd7-v5nnq\" (UID: \"51a79347-9247-46d7-b27d-e6826d9c947e\") " pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.591651 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.591632 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5755r\" (UniqueName: \"kubernetes.io/projected/51a79347-9247-46d7-b27d-e6826d9c947e-kube-api-access-5755r\") pod \"maas-controller-696f68cd7-v5nnq\" (UID: \"51a79347-9247-46d7-b27d-e6826d9c947e\") " pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.671570 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.671543 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:46.802088 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:46.802065 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-696f68cd7-v5nnq"] Jul 05 13:54:46.804087 ip-10-0-142-97 kubenswrapper[2560]: W0705 13:54:46.804061 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a79347_9247_46d7_b27d_e6826d9c947e.slice/crio-a6d8dd657567ce5b2cce0b44276bfbe7cdfb195149ec321c17db553fe173cf24 WatchSource:0}: Error finding container a6d8dd657567ce5b2cce0b44276bfbe7cdfb195149ec321c17db553fe173cf24: Status 404 returned error can't find the container with id a6d8dd657567ce5b2cce0b44276bfbe7cdfb195149ec321c17db553fe173cf24 Jul 05 13:54:47.559123 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:47.559040 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-696f68cd7-v5nnq" event={"ID":"51a79347-9247-46d7-b27d-e6826d9c947e","Type":"ContainerStarted","Data":"562b484ce479d52be7b3ed86313fdd21ae08f3553886eb0c95fdd40b2ee57b54"} Jul 05 13:54:47.559123 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:47.559096 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-696f68cd7-v5nnq" event={"ID":"51a79347-9247-46d7-b27d-e6826d9c947e","Type":"ContainerStarted","Data":"a6d8dd657567ce5b2cce0b44276bfbe7cdfb195149ec321c17db553fe173cf24"} Jul 05 13:54:47.559561 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:47.559181 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:54:47.600669 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:47.600614 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-696f68cd7-v5nnq" podStartSLOduration=1.114911514 podStartE2EDuration="1.600599253s" totalImagesPullingTime="485.687739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:54:47.600061505 +0000 UTC m=+963.779903952" watchObservedRunningTime="2026-07-05 13:54:47.600599253 +0000 UTC m=+963.780441698" Jul 05 13:54:58.570859 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:54:58.570824 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-696f68cd7-v5nnq" Jul 05 13:55:08.480021 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:08.479936 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:24.783547 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:24.783496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:24.887829 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:24.887802 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:25.087003 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:25.086974 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:30.762901 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:30.762847 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:30.833152 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:30.833109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:30.937555 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:30.937522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:31.035225 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:31.035141 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:31.146645 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:31.146617 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:36.864094 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:36.864058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:55:36.964842 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:55:36.964799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:21.411582 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:58:21.411508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:33.932788 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:58:33.932760 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:34.033543 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:58:34.033502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 13:58:45.178488 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:58:45.178460 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:58:45.179867 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:58:45.179844 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 13:59:02.565821 ip-10-0-142-97 kubenswrapper[2560]: I0705 13:59:02.565789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:03:19.188031 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:03:19.187997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:03:20.662876 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:03:20.662842 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:03:28.555151 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:03:28.555111 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:03:35.842574 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:03:35.842541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:03:45.209338 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:03:45.209295 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:03:45.211667 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:03:45.211645 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:06:11.392730 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:11.392675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:11.471687 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:11.471656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:17.481618 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:17.481579 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:17.584739 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:17.584707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:17.912893 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:17.912864 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:24.320514 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:24.320482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:24.431106 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:24.431071 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:24.567227 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:24.567195 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:24.663683 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:24.663646 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:25.749785 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.749741 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:25.845158 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.845118 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:25.926090 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.926060 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr"] Jul 05 14:06:25.929257 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.929238 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:25.932357 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.932334 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-0ca40252-serving-cert\"" Jul 05 14:06:25.943529 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.943507 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr"] Jul 05 14:06:25.978298 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.978268 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:25.980350 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.980303 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c356736-2f96-41b1-af70-ee0043ebe672-openshift-service-ca\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:25.980430 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.980367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:25.980430 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:25.980397 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn99s\" (UniqueName: \"kubernetes.io/projected/8c356736-2f96-41b1-af70-ee0043ebe672-kube-api-access-dn99s\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.081648 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.081623 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn99s\" (UniqueName: \"kubernetes.io/projected/8c356736-2f96-41b1-af70-ee0043ebe672-kube-api-access-dn99s\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.081753 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.081706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c356736-2f96-41b1-af70-ee0043ebe672-openshift-service-ca\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.081753 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.081732 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.081840 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:06:26.081830 2560 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-0ca40252-serving-cert: secret "maas-api-e2e-ait-0ca40252-serving-cert" not found Jul 05 14:06:26.081907 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:06:26.081896 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls podName:8c356736-2f96-41b1-af70-ee0043ebe672 nodeName:}" failed. No retries permitted until 2026-07-05 14:06:26.581876642 +0000 UTC m=+1662.761719065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls") pod "maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" (UID: "8c356736-2f96-41b1-af70-ee0043ebe672") : secret "maas-api-e2e-ait-0ca40252-serving-cert" not found Jul 05 14:06:26.082398 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.082383 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c356736-2f96-41b1-af70-ee0043ebe672-openshift-service-ca\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.093008 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.092982 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn99s\" (UniqueName: \"kubernetes.io/projected/8c356736-2f96-41b1-af70-ee0043ebe672-kube-api-access-dn99s\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.285153 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.285123 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:26.455613 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.455533 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:26.584733 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.584699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.587159 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.587136 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls\") pod \"maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.626278 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.626252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:26.840914 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.840886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:26.974429 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.974404 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr"] Jul 05 14:06:26.976376 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:06:26.976347 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c356736_2f96_41b1_af70_ee0043ebe672.slice/crio-501e44a70cef16fe53743e7b0961802f0d050fe344f709dc8c447bd736c18256 WatchSource:0}: Error finding container 501e44a70cef16fe53743e7b0961802f0d050fe344f709dc8c447bd736c18256: Status 404 returned error can't find the container with id 501e44a70cef16fe53743e7b0961802f0d050fe344f709dc8c447bd736c18256 Jul 05 14:06:26.978273 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:26.978252 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 14:06:27.898770 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:27.898736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" event={"ID":"8c356736-2f96-41b1-af70-ee0043ebe672","Type":"ContainerStarted","Data":"34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50"} Jul 05 14:06:27.899159 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:27.898784 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" event={"ID":"8c356736-2f96-41b1-af70-ee0043ebe672","Type":"ContainerStarted","Data":"501e44a70cef16fe53743e7b0961802f0d050fe344f709dc8c447bd736c18256"} Jul 05 14:06:27.899159 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:27.898872 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:27.920768 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:27.920724 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" podStartSLOduration=2.497721318 podStartE2EDuration="2.920709366s" totalImagesPullingTime="422.988048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:06:27.920284993 +0000 UTC m=+1664.100127439" watchObservedRunningTime="2026-07-05 14:06:27.920709366 +0000 UTC m=+1664.100552062" Jul 05 14:06:27.951889 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:27.951853 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:30.993367 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:30.993295 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:31.140009 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.139972 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:31.243088 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.243045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:31.387680 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.387641 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr"] Jul 05 14:06:31.387904 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.387875 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" podUID="8c356736-2f96-41b1-af70-ee0043ebe672" containerName="maas-api" containerID="cri-o://34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50" gracePeriod=30 Jul 05 14:06:31.436706 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.436680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:31.624490 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.624463 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:31.728722 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.728661 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls\") pod \"8c356736-2f96-41b1-af70-ee0043ebe672\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " Jul 05 14:06:31.728722 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.728698 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn99s\" (UniqueName: \"kubernetes.io/projected/8c356736-2f96-41b1-af70-ee0043ebe672-kube-api-access-dn99s\") pod \"8c356736-2f96-41b1-af70-ee0043ebe672\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " Jul 05 14:06:31.728908 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.728722 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c356736-2f96-41b1-af70-ee0043ebe672-openshift-service-ca\") pod \"8c356736-2f96-41b1-af70-ee0043ebe672\" (UID: \"8c356736-2f96-41b1-af70-ee0043ebe672\") " Jul 05 14:06:31.729120 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.729097 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c356736-2f96-41b1-af70-ee0043ebe672-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c356736-2f96-41b1-af70-ee0043ebe672" (UID: "8c356736-2f96-41b1-af70-ee0043ebe672"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:06:31.730743 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.730721 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c356736-2f96-41b1-af70-ee0043ebe672" (UID: "8c356736-2f96-41b1-af70-ee0043ebe672"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:06:31.730844 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.730791 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c356736-2f96-41b1-af70-ee0043ebe672-kube-api-access-dn99s" (OuterVolumeSpecName: "kube-api-access-dn99s") pod "8c356736-2f96-41b1-af70-ee0043ebe672" (UID: "8c356736-2f96-41b1-af70-ee0043ebe672"). InnerVolumeSpecName "kube-api-access-dn99s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:06:31.829655 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.829612 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn99s\" (UniqueName: \"kubernetes.io/projected/8c356736-2f96-41b1-af70-ee0043ebe672-kube-api-access-dn99s\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:06:31.829655 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.829653 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c356736-2f96-41b1-af70-ee0043ebe672-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:06:31.829655 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.829663 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c356736-2f96-41b1-af70-ee0043ebe672-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:06:31.922731 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.922698 2560 generic.go:356] "Generic (PLEG): container finished" podID="8c356736-2f96-41b1-af70-ee0043ebe672" containerID="34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50" exitCode=0 Jul 05 14:06:31.922731 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.922746 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" Jul 05 14:06:31.922998 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.922868 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" event={"ID":"8c356736-2f96-41b1-af70-ee0043ebe672","Type":"ContainerDied","Data":"34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50"} Jul 05 14:06:31.922998 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.922948 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr" event={"ID":"8c356736-2f96-41b1-af70-ee0043ebe672","Type":"ContainerDied","Data":"501e44a70cef16fe53743e7b0961802f0d050fe344f709dc8c447bd736c18256"} Jul 05 14:06:31.922998 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.922979 2560 scope.go:117] "RemoveContainer" containerID="34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50" Jul 05 14:06:31.933479 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.933463 2560 scope.go:117] "RemoveContainer" containerID="34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50" Jul 05 14:06:31.933722 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:06:31.933702 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50\": container with ID starting with 34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50 not found: ID does not exist" containerID="34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50" Jul 05 14:06:31.933784 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.933734 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50"} err="failed to get container status \"34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50\": rpc error: code = NotFound desc = could not find container \"34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50\": container with ID starting with 34cc79d9affacccd2f66f8cdfb52d7c5c9220e12703a6f08bbfba20a6a4eea50 not found: ID does not exist" Jul 05 14:06:31.947518 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.947497 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr"] Jul 05 14:06:31.952043 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:31.952018 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-0ca40252-5c9c5648c5-pwkxr"] Jul 05 14:06:33.590654 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:33.590614 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c356736-2f96-41b1-af70-ee0043ebe672" path="/var/lib/kubelet/pods/8c356736-2f96-41b1-af70-ee0043ebe672/volumes" Jul 05 14:06:34.170650 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:34.170613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:37.510290 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:37.510253 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:37.586696 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:37.586662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:37.755757 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:37.755711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:37.860231 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:37.860205 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:38.962135 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:38.962097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:39.050622 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.050592 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj"] Jul 05 14:06:39.051068 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.051050 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c356736-2f96-41b1-af70-ee0043ebe672" containerName="maas-api" Jul 05 14:06:39.051153 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.051072 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c356736-2f96-41b1-af70-ee0043ebe672" containerName="maas-api" Jul 05 14:06:39.051216 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.051178 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c356736-2f96-41b1-af70-ee0043ebe672" containerName="maas-api" Jul 05 14:06:39.055616 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.055594 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.059483 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.059464 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b8ce2e85-serving-cert\"" Jul 05 14:06:39.073407 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.073381 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj"] Jul 05 14:06:39.080669 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.080644 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:39.189859 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.189828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d165d6-1ce5-49c7-853e-ea65f6734149-openshift-service-ca\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.190013 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.189877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4ls5\" (UniqueName: \"kubernetes.io/projected/f9d165d6-1ce5-49c7-853e-ea65f6734149-kube-api-access-z4ls5\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.190013 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.189974 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d165d6-1ce5-49c7-853e-ea65f6734149-maas-api-tls\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.190736 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.190707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:39.291105 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.291037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d165d6-1ce5-49c7-853e-ea65f6734149-maas-api-tls\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.291105 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.291100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d165d6-1ce5-49c7-853e-ea65f6734149-openshift-service-ca\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.291265 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.291126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4ls5\" (UniqueName: \"kubernetes.io/projected/f9d165d6-1ce5-49c7-853e-ea65f6734149-kube-api-access-z4ls5\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.291862 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.291832 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d165d6-1ce5-49c7-853e-ea65f6734149-openshift-service-ca\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.293474 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.293454 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d165d6-1ce5-49c7-853e-ea65f6734149-maas-api-tls\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.310533 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.310504 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4ls5\" (UniqueName: \"kubernetes.io/projected/f9d165d6-1ce5-49c7-853e-ea65f6734149-kube-api-access-z4ls5\") pod \"maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.365992 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.365966 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.402159 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.402126 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:39.508433 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.508402 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj"] Jul 05 14:06:39.509745 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:06:39.509707 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d165d6_1ce5_49c7_853e_ea65f6734149.slice/crio-22c97147adedb3c269fcfb8db1c6f10121e0cc11f341f231dd8a4a69fa618cee WatchSource:0}: Error finding container 22c97147adedb3c269fcfb8db1c6f10121e0cc11f341f231dd8a4a69fa618cee: Status 404 returned error can't find the container with id 22c97147adedb3c269fcfb8db1c6f10121e0cc11f341f231dd8a4a69fa618cee Jul 05 14:06:39.751038 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.751011 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:39.849408 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.849383 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:39.966353 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.966298 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" event={"ID":"f9d165d6-1ce5-49c7-853e-ea65f6734149","Type":"ContainerStarted","Data":"062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866"} Jul 05 14:06:39.966686 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.966383 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:39.966686 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.966407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" event={"ID":"f9d165d6-1ce5-49c7-853e-ea65f6734149","Type":"ContainerStarted","Data":"22c97147adedb3c269fcfb8db1c6f10121e0cc11f341f231dd8a4a69fa618cee"} Jul 05 14:06:39.992704 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:39.992652 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" podStartSLOduration=0.63955946 podStartE2EDuration="992.638374ms" totalImagesPullingTime="353.078914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:06:39.991067212 +0000 UTC m=+1676.170909681" watchObservedRunningTime="2026-07-05 14:06:39.992638374 +0000 UTC m=+1676.172480821" Jul 05 14:06:41.169265 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:41.169237 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:44.218569 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.218529 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:44.576512 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.576483 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj"] Jul 05 14:06:44.576706 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.576684 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" podUID="f9d165d6-1ce5-49c7-853e-ea65f6734149" containerName="maas-api" containerID="cri-o://062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866" gracePeriod=30 Jul 05 14:06:44.582107 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.582085 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:44.606657 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.606603 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:44.820961 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.820939 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:44.938033 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.937969 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d165d6-1ce5-49c7-853e-ea65f6734149-openshift-service-ca\") pod \"f9d165d6-1ce5-49c7-853e-ea65f6734149\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " Jul 05 14:06:44.938165 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.938054 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4ls5\" (UniqueName: \"kubernetes.io/projected/f9d165d6-1ce5-49c7-853e-ea65f6734149-kube-api-access-z4ls5\") pod \"f9d165d6-1ce5-49c7-853e-ea65f6734149\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " Jul 05 14:06:44.938165 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.938111 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d165d6-1ce5-49c7-853e-ea65f6734149-maas-api-tls\") pod \"f9d165d6-1ce5-49c7-853e-ea65f6734149\" (UID: \"f9d165d6-1ce5-49c7-853e-ea65f6734149\") " Jul 05 14:06:44.938288 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.938263 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9d165d6-1ce5-49c7-853e-ea65f6734149-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9d165d6-1ce5-49c7-853e-ea65f6734149" (UID: "f9d165d6-1ce5-49c7-853e-ea65f6734149"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:06:44.940083 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.940055 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d165d6-1ce5-49c7-853e-ea65f6734149-kube-api-access-z4ls5" (OuterVolumeSpecName: "kube-api-access-z4ls5") pod "f9d165d6-1ce5-49c7-853e-ea65f6734149" (UID: "f9d165d6-1ce5-49c7-853e-ea65f6734149"). InnerVolumeSpecName "kube-api-access-z4ls5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:06:44.940083 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.940060 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9d165d6-1ce5-49c7-853e-ea65f6734149-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9d165d6-1ce5-49c7-853e-ea65f6734149" (UID: "f9d165d6-1ce5-49c7-853e-ea65f6734149"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:06:44.995037 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.995008 2560 generic.go:356] "Generic (PLEG): container finished" podID="f9d165d6-1ce5-49c7-853e-ea65f6734149" containerID="062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866" exitCode=0 Jul 05 14:06:44.995138 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.995060 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" Jul 05 14:06:44.995138 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.995096 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" event={"ID":"f9d165d6-1ce5-49c7-853e-ea65f6734149","Type":"ContainerDied","Data":"062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866"} Jul 05 14:06:44.995249 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.995153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj" event={"ID":"f9d165d6-1ce5-49c7-853e-ea65f6734149","Type":"ContainerDied","Data":"22c97147adedb3c269fcfb8db1c6f10121e0cc11f341f231dd8a4a69fa618cee"} Jul 05 14:06:44.995249 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:44.995179 2560 scope.go:117] "RemoveContainer" containerID="062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866" Jul 05 14:06:45.003680 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.003666 2560 scope.go:117] "RemoveContainer" containerID="062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866" Jul 05 14:06:45.003933 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:06:45.003916 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866\": container with ID starting with 062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866 not found: ID does not exist" containerID="062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866" Jul 05 14:06:45.003978 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.003942 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866"} err="failed to get container status \"062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866\": rpc error: code = NotFound desc = could not find container \"062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866\": container with ID starting with 062dcb023460777dfaf8736d36cb78049bef49b802b063a20bedf7db31d21866 not found: ID does not exist" Jul 05 14:06:45.019633 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.019604 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj"] Jul 05 14:06:45.027495 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.027476 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b8ce2e85-67bd8d7fb4-xg4tj"] Jul 05 14:06:45.038807 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.038791 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4ls5\" (UniqueName: \"kubernetes.io/projected/f9d165d6-1ce5-49c7-853e-ea65f6734149-kube-api-access-z4ls5\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:06:45.038872 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.038811 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9d165d6-1ce5-49c7-853e-ea65f6734149-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:06:45.038872 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.038821 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d165d6-1ce5-49c7-853e-ea65f6734149-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:06:45.591221 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:45.591188 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9d165d6-1ce5-49c7-853e-ea65f6734149" path="/var/lib/kubelet/pods/f9d165d6-1ce5-49c7-853e-ea65f6734149/volumes" Jul 05 14:06:47.128364 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:47.128330 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:47.205575 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:47.205549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:50.511503 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:50.511471 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:50.605033 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:50.605001 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:51.845195 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:51.845170 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:56.792715 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:56.792676 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:56.893589 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:56.893563 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:57.030079 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:57.030044 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:57.128210 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:57.128184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:58.232969 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.232931 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:58.332704 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.332675 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g"] Jul 05 14:06:58.333057 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.333043 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9d165d6-1ce5-49c7-853e-ea65f6734149" containerName="maas-api" Jul 05 14:06:58.333097 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.333059 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9d165d6-1ce5-49c7-853e-ea65f6734149" containerName="maas-api" Jul 05 14:06:58.333133 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.333121 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9d165d6-1ce5-49c7-853e-ea65f6734149" containerName="maas-api" Jul 05 14:06:58.336109 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.336092 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.339228 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.339206 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-cdb84f0d-serving-cert\"" Jul 05 14:06:58.339575 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.339549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:58.351040 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.351017 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g"] Jul 05 14:06:58.427668 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.427643 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:58.439603 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.439578 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl29w\" (UniqueName: \"kubernetes.io/projected/bd7b72d0-bbe6-4343-879e-4180105401c0-kube-api-access-dl29w\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.439727 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.439609 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7b72d0-bbe6-4343-879e-4180105401c0-openshift-service-ca\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.439727 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.439705 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.540761 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.540699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl29w\" (UniqueName: \"kubernetes.io/projected/bd7b72d0-bbe6-4343-879e-4180105401c0-kube-api-access-dl29w\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.540761 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.540740 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7b72d0-bbe6-4343-879e-4180105401c0-openshift-service-ca\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.540938 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.540781 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.540938 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:06:58.540882 2560 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-cdb84f0d-serving-cert: secret "maas-api-e2e-derive-cdb84f0d-serving-cert" not found Jul 05 14:06:58.541045 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:06:58.540941 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls podName:bd7b72d0-bbe6-4343-879e-4180105401c0 nodeName:}" failed. No retries permitted until 2026-07-05 14:06:59.040923746 +0000 UTC m=+1695.220766172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls") pod "maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" (UID: "bd7b72d0-bbe6-4343-879e-4180105401c0") : secret "maas-api-e2e-derive-cdb84f0d-serving-cert" not found Jul 05 14:06:58.541360 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.541342 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7b72d0-bbe6-4343-879e-4180105401c0-openshift-service-ca\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.550324 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.550288 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl29w\" (UniqueName: \"kubernetes.io/projected/bd7b72d0-bbe6-4343-879e-4180105401c0-kube-api-access-dl29w\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:58.797708 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:58.797628 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:59.031545 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:59.031509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:59.045452 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:59.045415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:59.047914 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:59.047852 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls\") pod \"maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:59.132976 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:59.132945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:06:59.246462 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:59.246433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:06:59.397659 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:06:59.397622 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g"] Jul 05 14:06:59.406748 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:06:59.406721 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7b72d0_bbe6_4343_879e_4180105401c0.slice/crio-1a26886c177a950fddb5f8cbb63d908d445daf69df7d04f28efaeba5b16440c5 WatchSource:0}: Error finding container 1a26886c177a950fddb5f8cbb63d908d445daf69df7d04f28efaeba5b16440c5: Status 404 returned error can't find the container with id 1a26886c177a950fddb5f8cbb63d908d445daf69df7d04f28efaeba5b16440c5 Jul 05 14:07:00.061616 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:00.061566 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:00.066662 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:00.066619 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" event={"ID":"bd7b72d0-bbe6-4343-879e-4180105401c0","Type":"ContainerStarted","Data":"611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe"} Jul 05 14:07:00.066797 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:00.066681 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" event={"ID":"bd7b72d0-bbe6-4343-879e-4180105401c0","Type":"ContainerStarted","Data":"1a26886c177a950fddb5f8cbb63d908d445daf69df7d04f28efaeba5b16440c5"} Jul 05 14:07:00.066797 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:00.066712 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:07:00.099169 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:00.099122 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" podStartSLOduration=1.778564141 podStartE2EDuration="2.099107117s" totalImagesPullingTime="320.542976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:06:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:07:00.09710525 +0000 UTC m=+1696.276947696" watchObservedRunningTime="2026-07-05 14:07:00.099107117 +0000 UTC m=+1696.278949563" Jul 05 14:07:02.850604 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:02.850567 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:03.006753 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.006711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:03.107628 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.107544 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:03.223246 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.223214 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g"] Jul 05 14:07:03.223506 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.223477 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" podUID="bd7b72d0-bbe6-4343-879e-4180105401c0" containerName="maas-api" containerID="cri-o://611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe" gracePeriod=30 Jul 05 14:07:03.257822 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.257793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:03.466767 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.466745 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:07:03.583588 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.583549 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls\") pod \"bd7b72d0-bbe6-4343-879e-4180105401c0\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " Jul 05 14:07:03.583776 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.583626 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7b72d0-bbe6-4343-879e-4180105401c0-openshift-service-ca\") pod \"bd7b72d0-bbe6-4343-879e-4180105401c0\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " Jul 05 14:07:03.583776 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.583748 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl29w\" (UniqueName: \"kubernetes.io/projected/bd7b72d0-bbe6-4343-879e-4180105401c0-kube-api-access-dl29w\") pod \"bd7b72d0-bbe6-4343-879e-4180105401c0\" (UID: \"bd7b72d0-bbe6-4343-879e-4180105401c0\") " Jul 05 14:07:03.583998 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.583970 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd7b72d0-bbe6-4343-879e-4180105401c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd7b72d0-bbe6-4343-879e-4180105401c0" (UID: "bd7b72d0-bbe6-4343-879e-4180105401c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:07:03.585636 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.585607 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7b72d0-bbe6-4343-879e-4180105401c0-kube-api-access-dl29w" (OuterVolumeSpecName: "kube-api-access-dl29w") pod "bd7b72d0-bbe6-4343-879e-4180105401c0" (UID: "bd7b72d0-bbe6-4343-879e-4180105401c0"). InnerVolumeSpecName "kube-api-access-dl29w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:07:03.585777 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.585760 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd7b72d0-bbe6-4343-879e-4180105401c0" (UID: "bd7b72d0-bbe6-4343-879e-4180105401c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:07:03.684688 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.684624 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl29w\" (UniqueName: \"kubernetes.io/projected/bd7b72d0-bbe6-4343-879e-4180105401c0-kube-api-access-dl29w\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:07:03.684688 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.684649 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7b72d0-bbe6-4343-879e-4180105401c0-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:07:03.684688 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:03.684661 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7b72d0-bbe6-4343-879e-4180105401c0-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:07:04.088500 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.088464 2560 generic.go:356] "Generic (PLEG): container finished" podID="bd7b72d0-bbe6-4343-879e-4180105401c0" containerID="611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe" exitCode=0 Jul 05 14:07:04.088906 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.088513 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" Jul 05 14:07:04.088906 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.088546 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" event={"ID":"bd7b72d0-bbe6-4343-879e-4180105401c0","Type":"ContainerDied","Data":"611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe"} Jul 05 14:07:04.088906 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.088602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g" event={"ID":"bd7b72d0-bbe6-4343-879e-4180105401c0","Type":"ContainerDied","Data":"1a26886c177a950fddb5f8cbb63d908d445daf69df7d04f28efaeba5b16440c5"} Jul 05 14:07:04.088906 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.088624 2560 scope.go:117] "RemoveContainer" containerID="611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe" Jul 05 14:07:04.096826 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.096804 2560 scope.go:117] "RemoveContainer" containerID="611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe" Jul 05 14:07:04.097098 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:07:04.097077 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe\": container with ID starting with 611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe not found: ID does not exist" containerID="611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe" Jul 05 14:07:04.097180 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.097103 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe"} err="failed to get container status \"611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe\": rpc error: code = NotFound desc = could not find container \"611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe\": container with ID starting with 611c952a6a510c16d36e599158be75b904b733d5492bb431ee81fe6ada389dfe not found: ID does not exist" Jul 05 14:07:04.108469 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.108448 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g"] Jul 05 14:07:04.113410 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:04.113388 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-cdb84f0d-f994bb9b6-jvj5g"] Jul 05 14:07:05.592592 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:05.592561 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd7b72d0-bbe6-4343-879e-4180105401c0" path="/var/lib/kubelet/pods/bd7b72d0-bbe6-4343-879e-4180105401c0/volumes" Jul 05 14:07:06.481134 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:06.481092 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:09.218280 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:09.218245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:09.310189 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:09.310132 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:09.724692 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:09.724659 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:09.833134 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:09.833106 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:09.963253 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:09.963218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:11.415012 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:11.414954 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:17.172424 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:17.172359 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:17.263664 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:17.263636 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:17.346968 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:17.346946 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:17.446908 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:17.446832 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:17.946186 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:17.946152 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:43.946855 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:43.946820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:50.060387 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:50.060352 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:50.164973 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:50.164939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:07:50.260294 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:07:50.260260 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:06.159606 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:06.159566 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:12.881784 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:12.881747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:13.044331 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:13.042302 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:13.221254 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:13.221171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:23.683144 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:23.683107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:24.780613 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:24.780567 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:30.324329 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:30.324278 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:30.432664 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:30.432632 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:30.528129 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:30.528099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:30.624666 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:30.624589 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:31.424324 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:31.424276 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:31.521512 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:31.521461 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:31.621111 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:31.621082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:33.282936 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:33.282903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:39.321532 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:39.321455 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:39.384836 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:39.384810 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:39.998511 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:39.998471 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:45.239872 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:45.239846 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:08:45.242400 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:45.242370 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:08:45.764412 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:45.764379 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:45.826103 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:45.826078 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:45.996545 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:45.996508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:46.306461 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:46.306421 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:46.945114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:46.945086 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:49.828798 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.828759 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:49.912534 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.912469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:49.913913 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.913892 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd"] Jul 05 14:08:49.914221 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.914208 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd7b72d0-bbe6-4343-879e-4180105401c0" containerName="maas-api" Jul 05 14:08:49.914336 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.914223 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd7b72d0-bbe6-4343-879e-4180105401c0" containerName="maas-api" Jul 05 14:08:49.914336 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.914283 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd7b72d0-bbe6-4343-879e-4180105401c0" containerName="maas-api" Jul 05 14:08:49.917299 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.917283 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:49.921059 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.921042 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-28ccab57-serving-cert\"" Jul 05 14:08:49.931324 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.931291 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd"] Jul 05 14:08:49.980239 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:49.980212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:50.015471 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.015447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b1dd110-c302-4d2b-873c-b08789720ab0-openshift-service-ca\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.015592 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.015486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.015592 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.015509 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t88mb\" (UniqueName: \"kubernetes.io/projected/8b1dd110-c302-4d2b-873c-b08789720ab0-kube-api-access-t88mb\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.116722 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.116653 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b1dd110-c302-4d2b-873c-b08789720ab0-openshift-service-ca\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.116722 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.116698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.116867 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.116726 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t88mb\" (UniqueName: \"kubernetes.io/projected/8b1dd110-c302-4d2b-873c-b08789720ab0-kube-api-access-t88mb\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.116867 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:08:50.116811 2560 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-28ccab57-serving-cert: secret "maas-api-e2e-mt-28ccab57-serving-cert" not found Jul 05 14:08:50.116941 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:08:50.116888 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls podName:8b1dd110-c302-4d2b-873c-b08789720ab0 nodeName:}" failed. No retries permitted until 2026-07-05 14:08:50.616866103 +0000 UTC m=+1806.796708530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls") pod "maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" (UID: "8b1dd110-c302-4d2b-873c-b08789720ab0") : secret "maas-api-e2e-mt-28ccab57-serving-cert" not found Jul 05 14:08:50.117396 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.117380 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b1dd110-c302-4d2b-873c-b08789720ab0-openshift-service-ca\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.128101 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.128080 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t88mb\" (UniqueName: \"kubernetes.io/projected/8b1dd110-c302-4d2b-873c-b08789720ab0-kube-api-access-t88mb\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.344184 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.344136 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:50.519258 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.519183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:50.619444 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.619417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.621968 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.621940 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls\") pod \"maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.622221 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.622195 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:50.826913 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.826853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:50.957061 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:50.957037 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd"] Jul 05 14:08:50.959067 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:08:50.959041 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1dd110_c302_4d2b_873c_b08789720ab0.slice/crio-3cb5ca8df0e0709b5565aa284059086d38872b39f0cb62daf7324f56190fcb73 WatchSource:0}: Error finding container 3cb5ca8df0e0709b5565aa284059086d38872b39f0cb62daf7324f56190fcb73: Status 404 returned error can't find the container with id 3cb5ca8df0e0709b5565aa284059086d38872b39f0cb62daf7324f56190fcb73 Jul 05 14:08:51.602204 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:51.602166 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" event={"ID":"8b1dd110-c302-4d2b-873c-b08789720ab0","Type":"ContainerStarted","Data":"21b9f12bf06437c776a67a87358b656289b1002fd3ade22b442bedc76ea294bc"} Jul 05 14:08:51.602350 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:51.602211 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" event={"ID":"8b1dd110-c302-4d2b-873c-b08789720ab0","Type":"ContainerStarted","Data":"3cb5ca8df0e0709b5565aa284059086d38872b39f0cb62daf7324f56190fcb73"} Jul 05 14:08:51.602350 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:51.602233 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:08:51.620800 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:51.620756 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" podStartSLOduration=2.117273822 podStartE2EDuration="2.620741458s" totalImagesPullingTime="503.467636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:08:51.618701167 +0000 UTC m=+1807.798543614" watchObservedRunningTime="2026-07-05 14:08:51.620741458 +0000 UTC m=+1807.800583904" Jul 05 14:08:54.357982 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:54.357949 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:55.019798 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:55.019758 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:55.112800 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:55.112761 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:55.431347 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:55.431292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:08:57.612987 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:08:57.612961 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:09:01.535439 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:01.535405 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:01.633837 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:01.633807 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:01.970363 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:01.970303 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:02.359555 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.359525 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:02.521826 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.521796 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd"] Jul 05 14:09:02.522301 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.522267 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" podUID="8b1dd110-c302-4d2b-873c-b08789720ab0" containerName="maas-api" containerID="cri-o://21b9f12bf06437c776a67a87358b656289b1002fd3ade22b442bedc76ea294bc" gracePeriod=30 Jul 05 14:09:02.552129 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.552102 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:02.607973 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.607930 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" podUID="8b1dd110-c302-4d2b-873c-b08789720ab0" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.42:8443/health\": dial tcp 10.134.0.42:8443: connect: connection refused" Jul 05 14:09:02.652044 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.652016 2560 generic.go:356] "Generic (PLEG): container finished" podID="8b1dd110-c302-4d2b-873c-b08789720ab0" containerID="21b9f12bf06437c776a67a87358b656289b1002fd3ade22b442bedc76ea294bc" exitCode=0 Jul 05 14:09:02.652195 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.652091 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" event={"ID":"8b1dd110-c302-4d2b-873c-b08789720ab0","Type":"ContainerDied","Data":"21b9f12bf06437c776a67a87358b656289b1002fd3ade22b442bedc76ea294bc"} Jul 05 14:09:02.762287 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.762264 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:09:02.821819 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.821789 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls\") pod \"8b1dd110-c302-4d2b-873c-b08789720ab0\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " Jul 05 14:09:02.821957 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.821855 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b1dd110-c302-4d2b-873c-b08789720ab0-openshift-service-ca\") pod \"8b1dd110-c302-4d2b-873c-b08789720ab0\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " Jul 05 14:09:02.821957 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.821888 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t88mb\" (UniqueName: \"kubernetes.io/projected/8b1dd110-c302-4d2b-873c-b08789720ab0-kube-api-access-t88mb\") pod \"8b1dd110-c302-4d2b-873c-b08789720ab0\" (UID: \"8b1dd110-c302-4d2b-873c-b08789720ab0\") " Jul 05 14:09:02.822221 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.822194 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b1dd110-c302-4d2b-873c-b08789720ab0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b1dd110-c302-4d2b-873c-b08789720ab0" (UID: "8b1dd110-c302-4d2b-873c-b08789720ab0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:09:02.823886 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.823862 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b1dd110-c302-4d2b-873c-b08789720ab0" (UID: "8b1dd110-c302-4d2b-873c-b08789720ab0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:09:02.824025 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.824009 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b1dd110-c302-4d2b-873c-b08789720ab0-kube-api-access-t88mb" (OuterVolumeSpecName: "kube-api-access-t88mb") pod "8b1dd110-c302-4d2b-873c-b08789720ab0" (UID: "8b1dd110-c302-4d2b-873c-b08789720ab0"). InnerVolumeSpecName "kube-api-access-t88mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:09:02.922726 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.922652 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b1dd110-c302-4d2b-873c-b08789720ab0-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:09:02.922726 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.922678 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b1dd110-c302-4d2b-873c-b08789720ab0-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:09:02.922726 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:02.922687 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t88mb\" (UniqueName: \"kubernetes.io/projected/8b1dd110-c302-4d2b-873c-b08789720ab0-kube-api-access-t88mb\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:09:03.657933 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:03.657908 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" Jul 05 14:09:03.658292 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:03.657951 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd" event={"ID":"8b1dd110-c302-4d2b-873c-b08789720ab0","Type":"ContainerDied","Data":"3cb5ca8df0e0709b5565aa284059086d38872b39f0cb62daf7324f56190fcb73"} Jul 05 14:09:03.658292 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:03.658005 2560 scope.go:117] "RemoveContainer" containerID="21b9f12bf06437c776a67a87358b656289b1002fd3ade22b442bedc76ea294bc" Jul 05 14:09:03.676795 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:03.676770 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd"] Jul 05 14:09:03.680873 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:03.680852 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-28ccab57-96cd76f6-xc7kd"] Jul 05 14:09:05.591215 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:05.591184 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b1dd110-c302-4d2b-873c-b08789720ab0" path="/var/lib/kubelet/pods/8b1dd110-c302-4d2b-873c-b08789720ab0/volumes" Jul 05 14:09:08.452089 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:08.452058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:08.513621 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:08.513594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:08.720422 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:08.720344 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:08.932349 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:08.932305 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:09.071286 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:09.071193 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:09.168763 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:09.168729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:09.369939 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:09.369908 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:14.880499 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:14.880466 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:14.988228 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:14.988197 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:15.080973 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:15.080944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:15.229353 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:15.229254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:21.125658 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:21.125618 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:21.187374 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:21.187348 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:27.853641 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:27.853605 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:28.169275 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:28.169190 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:28.924331 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:28.924286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:30.968495 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:30.968458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:31.044160 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.044128 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55"] Jul 05 14:09:31.044636 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.044618 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b1dd110-c302-4d2b-873c-b08789720ab0" containerName="maas-api" Jul 05 14:09:31.044636 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.044637 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b1dd110-c302-4d2b-873c-b08789720ab0" containerName="maas-api" Jul 05 14:09:31.044777 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.044716 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b1dd110-c302-4d2b-873c-b08789720ab0" containerName="maas-api" Jul 05 14:09:31.046195 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.046170 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:31.047809 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.047792 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.050499 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.050478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-2aab95e7-serving-cert\"" Jul 05 14:09:31.052974 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.052953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brbbz\" (UniqueName: \"kubernetes.io/projected/be291226-807e-400f-9a09-c4586562f1aa-kube-api-access-brbbz\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.053077 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.052992 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be291226-807e-400f-9a09-c4586562f1aa-openshift-service-ca\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.053147 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.053112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.059891 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.059870 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55"] Jul 05 14:09:31.117041 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.117014 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:31.153793 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.153768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.153935 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.153813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbbz\" (UniqueName: \"kubernetes.io/projected/be291226-807e-400f-9a09-c4586562f1aa-kube-api-access-brbbz\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.153935 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.153845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be291226-807e-400f-9a09-c4586562f1aa-openshift-service-ca\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.153935 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:09:31.153890 2560 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-2aab95e7-serving-cert: secret "maas-api-e2e-mt-2aab95e7-serving-cert" not found Jul 05 14:09:31.154073 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:09:31.153953 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls podName:be291226-807e-400f-9a09-c4586562f1aa nodeName:}" failed. No retries permitted until 2026-07-05 14:09:31.653939013 +0000 UTC m=+1847.833781437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls") pod "maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" (UID: "be291226-807e-400f-9a09-c4586562f1aa") : secret "maas-api-e2e-mt-2aab95e7-serving-cert" not found Jul 05 14:09:31.154470 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.154453 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be291226-807e-400f-9a09-c4586562f1aa-openshift-service-ca\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.166843 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.166824 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbbz\" (UniqueName: \"kubernetes.io/projected/be291226-807e-400f-9a09-c4586562f1aa-kube-api-access-brbbz\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.229457 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.229391 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:31.658270 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.658238 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.660813 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.660782 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls\") pod \"maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:31.765520 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.765489 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:31.867972 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.867939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:31.958512 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:31.958442 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:32.096439 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:32.096412 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55"] Jul 05 14:09:32.097839 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:09:32.097815 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe291226_807e_400f_9a09_c4586562f1aa.slice/crio-679967a378ef0a064860748441ab0c1195d54742d59f5a4f804f05deb424560c WatchSource:0}: Error finding container 679967a378ef0a064860748441ab0c1195d54742d59f5a4f804f05deb424560c: Status 404 returned error can't find the container with id 679967a378ef0a064860748441ab0c1195d54742d59f5a4f804f05deb424560c Jul 05 14:09:32.804139 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:32.804046 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" event={"ID":"be291226-807e-400f-9a09-c4586562f1aa","Type":"ContainerStarted","Data":"370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47"} Jul 05 14:09:32.804139 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:32.804102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" event={"ID":"be291226-807e-400f-9a09-c4586562f1aa","Type":"ContainerStarted","Data":"679967a378ef0a064860748441ab0c1195d54742d59f5a4f804f05deb424560c"} Jul 05 14:09:32.804387 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:32.804154 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:09:32.820243 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:32.820187 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" podStartSLOduration=1.511007113 podStartE2EDuration="1.820171613s" totalImagesPullingTime="309.1645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:09:32.819352277 +0000 UTC m=+1848.999194727" watchObservedRunningTime="2026-07-05 14:09:32.820171613 +0000 UTC m=+1849.000014059" Jul 05 14:09:35.471738 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:35.471704 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:09:38.815189 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:09:38.815157 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:10:07.583484 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:07.583395 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:07.683836 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:07.683793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:09.099015 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:09.098982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:28.399915 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:28.399881 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:28.926582 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:28.926543 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:29.027100 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.027068 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:29.183340 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.183241 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55"] Jul 05 14:10:29.183476 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.183456 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" podUID="be291226-807e-400f-9a09-c4586562f1aa" containerName="maas-api" containerID="cri-o://370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47" gracePeriod=30 Jul 05 14:10:29.199832 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.199811 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:29.430247 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.430225 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:10:29.609278 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.609251 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be291226-807e-400f-9a09-c4586562f1aa-openshift-service-ca\") pod \"be291226-807e-400f-9a09-c4586562f1aa\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " Jul 05 14:10:29.609459 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.609357 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls\") pod \"be291226-807e-400f-9a09-c4586562f1aa\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " Jul 05 14:10:29.609459 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.609410 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brbbz\" (UniqueName: \"kubernetes.io/projected/be291226-807e-400f-9a09-c4586562f1aa-kube-api-access-brbbz\") pod \"be291226-807e-400f-9a09-c4586562f1aa\" (UID: \"be291226-807e-400f-9a09-c4586562f1aa\") " Jul 05 14:10:29.609667 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.609649 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be291226-807e-400f-9a09-c4586562f1aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be291226-807e-400f-9a09-c4586562f1aa" (UID: "be291226-807e-400f-9a09-c4586562f1aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:10:29.611353 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.611325 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be291226-807e-400f-9a09-c4586562f1aa" (UID: "be291226-807e-400f-9a09-c4586562f1aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:10:29.611473 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.611394 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be291226-807e-400f-9a09-c4586562f1aa-kube-api-access-brbbz" (OuterVolumeSpecName: "kube-api-access-brbbz") pod "be291226-807e-400f-9a09-c4586562f1aa" (UID: "be291226-807e-400f-9a09-c4586562f1aa"). InnerVolumeSpecName "kube-api-access-brbbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:10:29.710582 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.710561 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be291226-807e-400f-9a09-c4586562f1aa-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:10:29.710582 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.710582 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be291226-807e-400f-9a09-c4586562f1aa-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:10:29.710708 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:29.710592 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brbbz\" (UniqueName: \"kubernetes.io/projected/be291226-807e-400f-9a09-c4586562f1aa-kube-api-access-brbbz\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:10:30.077524 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.077494 2560 generic.go:356] "Generic (PLEG): container finished" podID="be291226-807e-400f-9a09-c4586562f1aa" containerID="370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47" exitCode=0 Jul 05 14:10:30.077662 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.077540 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" Jul 05 14:10:30.077662 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.077533 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" event={"ID":"be291226-807e-400f-9a09-c4586562f1aa","Type":"ContainerDied","Data":"370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47"} Jul 05 14:10:30.077662 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.077656 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55" event={"ID":"be291226-807e-400f-9a09-c4586562f1aa","Type":"ContainerDied","Data":"679967a378ef0a064860748441ab0c1195d54742d59f5a4f804f05deb424560c"} Jul 05 14:10:30.077782 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.077677 2560 scope.go:117] "RemoveContainer" containerID="370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47" Jul 05 14:10:30.086591 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.086574 2560 scope.go:117] "RemoveContainer" containerID="370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47" Jul 05 14:10:30.086831 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:10:30.086811 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47\": container with ID starting with 370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47 not found: ID does not exist" containerID="370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47" Jul 05 14:10:30.086891 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.086842 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47"} err="failed to get container status \"370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47\": rpc error: code = NotFound desc = could not find container \"370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47\": container with ID starting with 370519a2b7f00c86a2686d2e36fd7ecbfafee84e5a1a00458bb2639e7a8e9a47 not found: ID does not exist" Jul 05 14:10:30.105044 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.105021 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55"] Jul 05 14:10:30.109786 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:30.109765 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-2aab95e7-675cb479c7-fwc55"] Jul 05 14:10:31.591014 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:31.590979 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be291226-807e-400f-9a09-c4586562f1aa" path="/var/lib/kubelet/pods/be291226-807e-400f-9a09-c4586562f1aa/volumes" Jul 05 14:10:34.430422 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:34.430382 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:10:39.564319 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:10:39.564283 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:15.669930 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:15.669896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:16.514734 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:16.514705 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:18.939888 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:18.939851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:18.991116 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:18.991084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:22.245655 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:22.245614 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:38.249528 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:38.249443 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:38.350513 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:38.350468 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:38.572208 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:38.572119 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:38.801918 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:38.801877 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:39.866692 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:39.866660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:39.956265 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:39.956224 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:40.062351 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:40.062290 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:40.843397 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:40.843353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:40.944681 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:40.944648 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:41.058754 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:41.058720 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:42.535844 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:42.535800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:49.208415 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:49.208380 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:49.304434 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:49.304397 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:49.919490 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:49.919452 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:56.251516 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:56.251475 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:56.424214 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:56.424178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:11:56.579609 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:11:56.579580 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:07.076959 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:07.076924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:07.400261 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:07.400172 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:28.371995 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:28.371955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:34.410666 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:34.410628 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:34.512590 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:34.512561 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:34.750697 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:34.750594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:35.079288 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:35.079254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:35.354231 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:35.354148 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:37.952550 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:37.952518 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:38.007743 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.007706 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6"] Jul 05 14:12:38.008031 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.008018 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be291226-807e-400f-9a09-c4586562f1aa" containerName="maas-api" Jul 05 14:12:38.008031 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.008032 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="be291226-807e-400f-9a09-c4586562f1aa" containerName="maas-api" Jul 05 14:12:38.008157 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.008095 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="be291226-807e-400f-9a09-c4586562f1aa" containerName="maas-api" Jul 05 14:12:38.011036 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.011009 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.014015 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.013994 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-2d7b22-serving-cert\"" Jul 05 14:12:38.023441 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.023420 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6"] Jul 05 14:12:38.045543 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.045519 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:38.121047 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.121015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:38.125152 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.125123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dece8-456d-4349-91b4-9f0e6c77b651-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.125270 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.125163 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnq75\" (UniqueName: \"kubernetes.io/projected/5f1dece8-456d-4349-91b4-9f0e6c77b651-kube-api-access-mnq75\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.125270 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.125219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.226366 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.226268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.226366 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.226363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dece8-456d-4349-91b4-9f0e6c77b651-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.226575 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.226383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnq75\" (UniqueName: \"kubernetes.io/projected/5f1dece8-456d-4349-91b4-9f0e6c77b651-kube-api-access-mnq75\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.226575 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:12:38.226422 2560 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-2d7b22-serving-cert: secret "maas-api-e2e-inf-a-2d7b22-serving-cert" not found Jul 05 14:12:38.226575 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:12:38.226497 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls podName:5f1dece8-456d-4349-91b4-9f0e6c77b651 nodeName:}" failed. No retries permitted until 2026-07-05 14:12:38.726479503 +0000 UTC m=+2034.906321927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls") pod "maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" (UID: "5f1dece8-456d-4349-91b4-9f0e6c77b651") : secret "maas-api-e2e-inf-a-2d7b22-serving-cert" not found Jul 05 14:12:38.227042 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.227020 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dece8-456d-4349-91b4-9f0e6c77b651-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.237855 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.237830 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnq75\" (UniqueName: \"kubernetes.io/projected/5f1dece8-456d-4349-91b4-9f0e6c77b651-kube-api-access-mnq75\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.537700 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.537627 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:38.730342 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.730298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.732681 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.732661 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:38.753060 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.753028 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:38.849721 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.849677 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:38.922145 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:38.922119 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:39.060962 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:39.060934 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6"] Jul 05 14:12:39.062565 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:12:39.062534 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1dece8_456d_4349_91b4_9f0e6c77b651.slice/crio-2126d7eca5c913214fabc09b22d5808962331520bbb6fd039756984bf45773a1 WatchSource:0}: Error finding container 2126d7eca5c913214fabc09b22d5808962331520bbb6fd039756984bf45773a1: Status 404 returned error can't find the container with id 2126d7eca5c913214fabc09b22d5808962331520bbb6fd039756984bf45773a1 Jul 05 14:12:39.064211 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:39.064191 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 14:12:39.695691 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:39.695659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" event={"ID":"5f1dece8-456d-4349-91b4-9f0e6c77b651","Type":"ContainerStarted","Data":"0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b"} Jul 05 14:12:39.695852 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:39.695717 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" event={"ID":"5f1dece8-456d-4349-91b4-9f0e6c77b651","Type":"ContainerStarted","Data":"2126d7eca5c913214fabc09b22d5808962331520bbb6fd039756984bf45773a1"} Jul 05 14:12:39.695852 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:39.695785 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:39.715063 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:39.715020 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" podStartSLOduration=2.295449764 podStartE2EDuration="2.715008312s" totalImagesPullingTime="419.558548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:12:39.712961958 +0000 UTC m=+2035.892804406" watchObservedRunningTime="2026-07-05 14:12:39.715008312 +0000 UTC m=+2035.894850762" Jul 05 14:12:42.421965 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:42.421929 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:42.760223 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:42.760139 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:43.050072 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:43.049997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:43.471221 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:43.471186 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:45.705652 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:45.705622 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:12:46.049347 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.049242 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:46.119168 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.119135 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp"] Jul 05 14:12:46.123108 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.123086 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.125464 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.125445 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-7c0371-serving-cert\"" Jul 05 14:12:46.138020 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.137998 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp"] Jul 05 14:12:46.139907 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.139882 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:46.226363 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.226292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:46.292076 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.292041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f473bd8-a827-442a-9477-9326a1a47a4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.292076 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.292075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f473bd8-a827-442a-9477-9326a1a47a4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.292256 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.292105 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzjlx\" (UniqueName: \"kubernetes.io/projected/4f473bd8-a827-442a-9477-9326a1a47a4f-kube-api-access-lzjlx\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.323857 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.323832 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:46.393068 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.393044 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f473bd8-a827-442a-9477-9326a1a47a4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.393194 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.393073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f473bd8-a827-442a-9477-9326a1a47a4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.393194 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.393095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzjlx\" (UniqueName: \"kubernetes.io/projected/4f473bd8-a827-442a-9477-9326a1a47a4f-kube-api-access-lzjlx\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.393692 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.393674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f473bd8-a827-442a-9477-9326a1a47a4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.395547 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.395531 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f473bd8-a827-442a-9477-9326a1a47a4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.402666 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.402643 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzjlx\" (UniqueName: \"kubernetes.io/projected/4f473bd8-a827-442a-9477-9326a1a47a4f-kube-api-access-lzjlx\") pod \"maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.437536 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.437508 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:46.610024 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.609999 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp"] Jul 05 14:12:46.611343 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:12:46.611305 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f473bd8_a827_442a_9477_9326a1a47a4f.slice/crio-ea3235e675ad0b32ec178c5c7ecdf1058e0815e5f4dca137692d18afd82e6f8c WatchSource:0}: Error finding container ea3235e675ad0b32ec178c5c7ecdf1058e0815e5f4dca137692d18afd82e6f8c: Status 404 returned error can't find the container with id ea3235e675ad0b32ec178c5c7ecdf1058e0815e5f4dca137692d18afd82e6f8c Jul 05 14:12:46.729613 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.729577 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" event={"ID":"4f473bd8-a827-442a-9477-9326a1a47a4f","Type":"ContainerStarted","Data":"ea3235e675ad0b32ec178c5c7ecdf1058e0815e5f4dca137692d18afd82e6f8c"} Jul 05 14:12:46.765098 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.765064 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:46.861807 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:46.861704 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:47.738304 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:47.738269 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" event={"ID":"4f473bd8-a827-442a-9477-9326a1a47a4f","Type":"ContainerStarted","Data":"cd705dbf30cfea8f165f1a118e248807813bf3d8cb903c3c607d3b20d08cff67"} Jul 05 14:12:47.738694 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:47.738358 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:12:47.756995 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:47.756945 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" podStartSLOduration=1.4028318020000001 podStartE2EDuration="1.756928598s" totalImagesPullingTime="354.096796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:12:47.75598253 +0000 UTC m=+2043.935824976" watchObservedRunningTime="2026-07-05 14:12:47.756928598 +0000 UTC m=+2043.936771043" Jul 05 14:12:50.423768 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:50.423697 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:12:53.748538 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:12:53.748512 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:13:00.443168 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:00.443122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:13:00.543503 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:00.543463 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:13:04.514636 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:04.514594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:13:07.573017 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:07.572921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:13:07.681388 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:07.681350 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:13:32.807055 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:32.807011 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:13:45.277555 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:45.277526 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:13:45.281808 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:45.281790 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:13:59.205898 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:13:59.205858 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:00.213680 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:00.213643 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:00.393407 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:00.393371 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:00.812192 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:00.812155 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:00.970829 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:00.970790 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:01.071570 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.071471 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:01.233294 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.233259 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6"] Jul 05 14:14:01.233738 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.233554 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" podUID="5f1dece8-456d-4349-91b4-9f0e6c77b651" containerName="maas-api" containerID="cri-o://0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b" gracePeriod=30 Jul 05 14:14:01.263937 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.263912 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:01.689166 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.689143 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:14:01.796288 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.796218 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dece8-456d-4349-91b4-9f0e6c77b651-openshift-service-ca\") pod \"5f1dece8-456d-4349-91b4-9f0e6c77b651\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " Jul 05 14:14:01.796425 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.796336 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnq75\" (UniqueName: \"kubernetes.io/projected/5f1dece8-456d-4349-91b4-9f0e6c77b651-kube-api-access-mnq75\") pod \"5f1dece8-456d-4349-91b4-9f0e6c77b651\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " Jul 05 14:14:01.796425 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.796361 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls\") pod \"5f1dece8-456d-4349-91b4-9f0e6c77b651\" (UID: \"5f1dece8-456d-4349-91b4-9f0e6c77b651\") " Jul 05 14:14:01.796641 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.796614 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f1dece8-456d-4349-91b4-9f0e6c77b651-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f1dece8-456d-4349-91b4-9f0e6c77b651" (UID: "5f1dece8-456d-4349-91b4-9f0e6c77b651"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:14:01.798335 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.798293 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f1dece8-456d-4349-91b4-9f0e6c77b651" (UID: "5f1dece8-456d-4349-91b4-9f0e6c77b651"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:14:01.798416 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.798342 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f1dece8-456d-4349-91b4-9f0e6c77b651-kube-api-access-mnq75" (OuterVolumeSpecName: "kube-api-access-mnq75") pod "5f1dece8-456d-4349-91b4-9f0e6c77b651" (UID: "5f1dece8-456d-4349-91b4-9f0e6c77b651"). InnerVolumeSpecName "kube-api-access-mnq75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:14:01.896885 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.896861 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnq75\" (UniqueName: \"kubernetes.io/projected/5f1dece8-456d-4349-91b4-9f0e6c77b651-kube-api-access-mnq75\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:14:01.896885 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.896882 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1dece8-456d-4349-91b4-9f0e6c77b651-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:14:01.897003 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:01.896891 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1dece8-456d-4349-91b4-9f0e6c77b651-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:14:02.091509 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.091476 2560 generic.go:356] "Generic (PLEG): container finished" podID="5f1dece8-456d-4349-91b4-9f0e6c77b651" containerID="0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b" exitCode=0 Jul 05 14:14:02.091639 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.091520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" event={"ID":"5f1dece8-456d-4349-91b4-9f0e6c77b651","Type":"ContainerDied","Data":"0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b"} Jul 05 14:14:02.091639 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.091576 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" event={"ID":"5f1dece8-456d-4349-91b4-9f0e6c77b651","Type":"ContainerDied","Data":"2126d7eca5c913214fabc09b22d5808962331520bbb6fd039756984bf45773a1"} Jul 05 14:14:02.091639 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.091597 2560 scope.go:117] "RemoveContainer" containerID="0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b" Jul 05 14:14:02.091639 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.091527 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6" Jul 05 14:14:02.100797 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.100772 2560 scope.go:117] "RemoveContainer" containerID="0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b" Jul 05 14:14:02.101057 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:14:02.101039 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b\": container with ID starting with 0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b not found: ID does not exist" containerID="0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b" Jul 05 14:14:02.101111 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.101066 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b"} err="failed to get container status \"0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b\": rpc error: code = NotFound desc = could not find container \"0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b\": container with ID starting with 0ff6541322048e6df44ef61c120d9155e5efeb459acc7df6752b0db85531a60b not found: ID does not exist" Jul 05 14:14:02.114566 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.114539 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6"] Jul 05 14:14:02.118531 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:02.118502 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-2d7b22-6c5c6bc694-9lnk6"] Jul 05 14:14:03.591231 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:03.591180 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f1dece8-456d-4349-91b4-9f0e6c77b651" path="/var/lib/kubelet/pods/5f1dece8-456d-4349-91b4-9f0e6c77b651/volumes" Jul 05 14:14:05.869971 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:05.869928 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:05.972529 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:05.972500 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:42.782188 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:42.782099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:43.126928 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:43.126894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:43.300194 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:43.300146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:43.474206 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:43.474117 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:43.894029 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:43.893995 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:43.992990 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:43.992958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:44.094842 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.094804 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:44.152005 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.151920 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp"] Jul 05 14:14:44.152168 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.152118 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" podUID="4f473bd8-a827-442a-9477-9326a1a47a4f" containerName="maas-api" containerID="cri-o://cd705dbf30cfea8f165f1a118e248807813bf3d8cb903c3c607d3b20d08cff67" gracePeriod=30 Jul 05 14:14:44.177694 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.177667 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:44.299081 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.299048 2560 generic.go:356] "Generic (PLEG): container finished" podID="4f473bd8-a827-442a-9477-9326a1a47a4f" containerID="cd705dbf30cfea8f165f1a118e248807813bf3d8cb903c3c607d3b20d08cff67" exitCode=0 Jul 05 14:14:44.299228 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.299116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" event={"ID":"4f473bd8-a827-442a-9477-9326a1a47a4f","Type":"ContainerDied","Data":"cd705dbf30cfea8f165f1a118e248807813bf3d8cb903c3c607d3b20d08cff67"} Jul 05 14:14:44.392826 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.392801 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:14:44.551134 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.551049 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f473bd8-a827-442a-9477-9326a1a47a4f-openshift-service-ca\") pod \"4f473bd8-a827-442a-9477-9326a1a47a4f\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " Jul 05 14:14:44.551134 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.551101 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f473bd8-a827-442a-9477-9326a1a47a4f-maas-api-tls\") pod \"4f473bd8-a827-442a-9477-9326a1a47a4f\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " Jul 05 14:14:44.551347 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.551198 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzjlx\" (UniqueName: \"kubernetes.io/projected/4f473bd8-a827-442a-9477-9326a1a47a4f-kube-api-access-lzjlx\") pod \"4f473bd8-a827-442a-9477-9326a1a47a4f\" (UID: \"4f473bd8-a827-442a-9477-9326a1a47a4f\") " Jul 05 14:14:44.551523 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.551493 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f473bd8-a827-442a-9477-9326a1a47a4f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f473bd8-a827-442a-9477-9326a1a47a4f" (UID: "4f473bd8-a827-442a-9477-9326a1a47a4f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:14:44.553229 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.553210 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f473bd8-a827-442a-9477-9326a1a47a4f-kube-api-access-lzjlx" (OuterVolumeSpecName: "kube-api-access-lzjlx") pod "4f473bd8-a827-442a-9477-9326a1a47a4f" (UID: "4f473bd8-a827-442a-9477-9326a1a47a4f"). InnerVolumeSpecName "kube-api-access-lzjlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:14:44.553352 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.553258 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f473bd8-a827-442a-9477-9326a1a47a4f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f473bd8-a827-442a-9477-9326a1a47a4f" (UID: "4f473bd8-a827-442a-9477-9326a1a47a4f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:14:44.652215 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.652188 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzjlx\" (UniqueName: \"kubernetes.io/projected/4f473bd8-a827-442a-9477-9326a1a47a4f-kube-api-access-lzjlx\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:14:44.652215 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.652212 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f473bd8-a827-442a-9477-9326a1a47a4f-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:14:44.652215 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:44.652222 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f473bd8-a827-442a-9477-9326a1a47a4f-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:14:45.305377 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:45.305352 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" Jul 05 14:14:45.305803 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:45.305384 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp" event={"ID":"4f473bd8-a827-442a-9477-9326a1a47a4f","Type":"ContainerDied","Data":"ea3235e675ad0b32ec178c5c7ecdf1058e0815e5f4dca137692d18afd82e6f8c"} Jul 05 14:14:45.305803 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:45.305446 2560 scope.go:117] "RemoveContainer" containerID="cd705dbf30cfea8f165f1a118e248807813bf3d8cb903c3c607d3b20d08cff67" Jul 05 14:14:45.328985 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:45.328964 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp"] Jul 05 14:14:45.333244 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:45.333223 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7c0371-5c9bc9f595-74hfp"] Jul 05 14:14:45.591045 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:45.590966 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f473bd8-a827-442a-9477-9326a1a47a4f" path="/var/lib/kubelet/pods/4f473bd8-a827-442a-9477-9326a1a47a4f/volumes" Jul 05 14:14:49.660438 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:49.660410 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:49.720270 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:49.720243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:49.948699 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:49.948584 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:50.164904 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:50.164866 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:50.509287 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:50.509257 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:50.811152 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:50.811069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:51.051167 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:51.051130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:53.712403 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.712364 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:53.788406 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.788375 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh"] Jul 05 14:14:53.789354 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.789330 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f473bd8-a827-442a-9477-9326a1a47a4f" containerName="maas-api" Jul 05 14:14:53.789487 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.789476 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f473bd8-a827-442a-9477-9326a1a47a4f" containerName="maas-api" Jul 05 14:14:53.789606 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.789595 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1dece8-456d-4349-91b4-9f0e6c77b651" containerName="maas-api" Jul 05 14:14:53.789684 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.789675 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1dece8-456d-4349-91b4-9f0e6c77b651" containerName="maas-api" Jul 05 14:14:53.789901 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.789889 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f1dece8-456d-4349-91b4-9f0e6c77b651" containerName="maas-api" Jul 05 14:14:53.789994 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.789984 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f473bd8-a827-442a-9477-9326a1a47a4f" containerName="maas-api" Jul 05 14:14:53.793619 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.793599 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.797267 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.797244 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-740454-serving-cert\"" Jul 05 14:14:53.800471 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.800449 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh"] Jul 05 14:14:53.807235 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.807210 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:53.824575 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.824554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-maas-api-tls\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.824659 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.824593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grt2l\" (UniqueName: \"kubernetes.io/projected/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-kube-api-access-grt2l\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.824659 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.824624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.893517 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.893488 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:53.925217 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.925195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grt2l\" (UniqueName: \"kubernetes.io/projected/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-kube-api-access-grt2l\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.925373 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.925237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.925373 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.925287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-maas-api-tls\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.925953 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.925902 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.927940 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.927915 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-maas-api-tls\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.937107 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.937083 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grt2l\" (UniqueName: \"kubernetes.io/projected/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-kube-api-access-grt2l\") pod \"maas-api-e2e-api-a-740454-b669cb6bb-m27sh\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:53.988138 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:53.988072 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:54.107465 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:54.107435 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:54.238648 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:54.238619 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh"] Jul 05 14:14:54.240847 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:14:54.240820 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e49f07_1279_46b2_a7dc_ccc1c5c3b5eb.slice/crio-8d5df37b4acf8e11df153dd4e4ec80906050212b3c814b411862f11b4fa02196 WatchSource:0}: Error finding container 8d5df37b4acf8e11df153dd4e4ec80906050212b3c814b411862f11b4fa02196: Status 404 returned error can't find the container with id 8d5df37b4acf8e11df153dd4e4ec80906050212b3c814b411862f11b4fa02196 Jul 05 14:14:54.270245 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:54.270212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:54.349994 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:54.349961 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" event={"ID":"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb","Type":"ContainerStarted","Data":"8d5df37b4acf8e11df153dd4e4ec80906050212b3c814b411862f11b4fa02196"} Jul 05 14:14:54.370798 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:54.370772 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:55.356465 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:55.356432 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" event={"ID":"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb","Type":"ContainerStarted","Data":"541a6d3465f02302e643eaf389481ebd4c8d8fe5c850e8967d163e516f9514f0"} Jul 05 14:14:55.356883 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:55.356541 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:14:55.376461 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:55.376413 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" podStartSLOduration=1.975509229 podStartE2EDuration="2.376400158s" totalImagesPullingTime="400.890929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:14:55.373846261 +0000 UTC m=+2171.553688708" watchObservedRunningTime="2026-07-05 14:14:55.376400158 +0000 UTC m=+2171.556242604" Jul 05 14:14:58.144176 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:58.144144 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:58.622968 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:58.622932 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:58.945400 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:58.945300 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:14:59.104787 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:14:59.104756 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:01.367950 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.367918 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:15:01.634060 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.633977 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:01.734474 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.734435 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:01.814955 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.814915 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd"] Jul 05 14:15:01.818539 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.818518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.821438 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.821406 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-9394f3-serving-cert\"" Jul 05 14:15:01.836896 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.836872 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd"] Jul 05 14:15:01.879011 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.878984 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:01.896107 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.896041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzckv\" (UniqueName: \"kubernetes.io/projected/48487ec7-e8d6-4741-bedc-472a49c37467-kube-api-access-lzckv\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.896107 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.896083 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48487ec7-e8d6-4741-bedc-472a49c37467-openshift-service-ca\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.896251 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.896205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48487ec7-e8d6-4741-bedc-472a49c37467-maas-api-tls\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.997250 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.997216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48487ec7-e8d6-4741-bedc-472a49c37467-maas-api-tls\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.997438 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.997263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzckv\" (UniqueName: \"kubernetes.io/projected/48487ec7-e8d6-4741-bedc-472a49c37467-kube-api-access-lzckv\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.997438 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.997294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48487ec7-e8d6-4741-bedc-472a49c37467-openshift-service-ca\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.997915 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.997897 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48487ec7-e8d6-4741-bedc-472a49c37467-openshift-service-ca\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:01.999680 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:01.999661 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48487ec7-e8d6-4741-bedc-472a49c37467-maas-api-tls\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:02.005589 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:02.005568 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzckv\" (UniqueName: \"kubernetes.io/projected/48487ec7-e8d6-4741-bedc-472a49c37467-kube-api-access-lzckv\") pod \"maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:02.027509 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:02.027481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:02.129434 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:02.129398 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:02.262277 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:02.262222 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd"] Jul 05 14:15:02.264698 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:15:02.264660 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48487ec7_e8d6_4741_bedc_472a49c37467.slice/crio-c485033fe6d71d6bb976cb21e9b68280eb61f0fffb7350e6f1618c49f048f7ec WatchSource:0}: Error finding container c485033fe6d71d6bb976cb21e9b68280eb61f0fffb7350e6f1618c49f048f7ec: Status 404 returned error can't find the container with id c485033fe6d71d6bb976cb21e9b68280eb61f0fffb7350e6f1618c49f048f7ec Jul 05 14:15:02.391451 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:02.391417 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" event={"ID":"48487ec7-e8d6-4741-bedc-472a49c37467","Type":"ContainerStarted","Data":"c485033fe6d71d6bb976cb21e9b68280eb61f0fffb7350e6f1618c49f048f7ec"} Jul 05 14:15:02.419297 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:02.419266 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:02.515305 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:02.515218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:03.397771 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:03.397737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" event={"ID":"48487ec7-e8d6-4741-bedc-472a49c37467","Type":"ContainerStarted","Data":"1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16"} Jul 05 14:15:03.398168 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:03.397859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:03.421326 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:03.421272 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" podStartSLOduration=2.088567052 podStartE2EDuration="2.421258737s" totalImagesPullingTime="332.691685ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:15:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:15:03.419901943 +0000 UTC m=+2179.599744390" watchObservedRunningTime="2026-07-05 14:15:03.421258737 +0000 UTC m=+2179.601101249" Jul 05 14:15:06.140758 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:06.140729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:08.477909 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:08.477868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:08.581400 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:08.581367 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:09.408638 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:09.408608 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:14.264780 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:14.264744 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:14.624150 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:14.624116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:14.991384 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:14.991271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:15.091142 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.091095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:15.233264 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.233232 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh"] Jul 05 14:15:15.233517 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.233492 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" podUID="f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" containerName="maas-api" containerID="cri-o://541a6d3465f02302e643eaf389481ebd4c8d8fe5c850e8967d163e516f9514f0" gracePeriod=30 Jul 05 14:15:15.261784 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.261747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:15.459581 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.459556 2560 generic.go:356] "Generic (PLEG): container finished" podID="f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" containerID="541a6d3465f02302e643eaf389481ebd4c8d8fe5c850e8967d163e516f9514f0" exitCode=0 Jul 05 14:15:15.459882 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.459601 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" event={"ID":"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb","Type":"ContainerDied","Data":"541a6d3465f02302e643eaf389481ebd4c8d8fe5c850e8967d163e516f9514f0"} Jul 05 14:15:15.459882 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.459642 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" event={"ID":"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb","Type":"ContainerDied","Data":"8d5df37b4acf8e11df153dd4e4ec80906050212b3c814b411862f11b4fa02196"} Jul 05 14:15:15.459882 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.459659 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d5df37b4acf8e11df153dd4e4ec80906050212b3c814b411862f11b4fa02196" Jul 05 14:15:15.469925 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.469906 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:15:15.620633 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.620609 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grt2l\" (UniqueName: \"kubernetes.io/projected/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-kube-api-access-grt2l\") pod \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " Jul 05 14:15:15.620785 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.620687 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-openshift-service-ca\") pod \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " Jul 05 14:15:15.620785 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.620719 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-maas-api-tls\") pod \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\" (UID: \"f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb\") " Jul 05 14:15:15.621019 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.620993 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" (UID: "f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:15:15.622654 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.622632 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" (UID: "f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:15:15.622719 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.622637 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-kube-api-access-grt2l" (OuterVolumeSpecName: "kube-api-access-grt2l") pod "f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" (UID: "f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb"). InnerVolumeSpecName "kube-api-access-grt2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:15:15.722176 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.722154 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:15:15.722176 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.722176 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:15:15.722353 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:15.722185 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grt2l\" (UniqueName: \"kubernetes.io/projected/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb-kube-api-access-grt2l\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:15:16.464401 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:16.464370 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh" Jul 05 14:15:16.486249 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:16.486226 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh"] Jul 05 14:15:16.490287 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:16.490264 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-740454-b669cb6bb-m27sh"] Jul 05 14:15:17.591225 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:17.591191 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" path="/var/lib/kubelet/pods/f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb/volumes" Jul 05 14:15:20.305850 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:20.305783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:20.406741 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:20.406709 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:20.561144 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:20.561063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:20.791433 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:20.791399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:20.960739 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:20.960704 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:21.343293 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:21.343253 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:21.483853 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:21.483823 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:21.586661 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:21.586636 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:21.747906 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:21.747828 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd"] Jul 05 14:15:21.748085 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:21.748061 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" podUID="48487ec7-e8d6-4741-bedc-472a49c37467" containerName="maas-api" containerID="cri-o://1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16" gracePeriod=30 Jul 05 14:15:21.855045 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:21.855020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:21.987163 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:21.987143 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:22.070828 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.070771 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48487ec7-e8d6-4741-bedc-472a49c37467-maas-api-tls\") pod \"48487ec7-e8d6-4741-bedc-472a49c37467\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " Jul 05 14:15:22.070828 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.070812 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzckv\" (UniqueName: \"kubernetes.io/projected/48487ec7-e8d6-4741-bedc-472a49c37467-kube-api-access-lzckv\") pod \"48487ec7-e8d6-4741-bedc-472a49c37467\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " Jul 05 14:15:22.070976 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.070832 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48487ec7-e8d6-4741-bedc-472a49c37467-openshift-service-ca\") pod \"48487ec7-e8d6-4741-bedc-472a49c37467\" (UID: \"48487ec7-e8d6-4741-bedc-472a49c37467\") " Jul 05 14:15:22.071208 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.071180 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48487ec7-e8d6-4741-bedc-472a49c37467-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48487ec7-e8d6-4741-bedc-472a49c37467" (UID: "48487ec7-e8d6-4741-bedc-472a49c37467"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:15:22.072875 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.072850 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48487ec7-e8d6-4741-bedc-472a49c37467-kube-api-access-lzckv" (OuterVolumeSpecName: "kube-api-access-lzckv") pod "48487ec7-e8d6-4741-bedc-472a49c37467" (UID: "48487ec7-e8d6-4741-bedc-472a49c37467"). InnerVolumeSpecName "kube-api-access-lzckv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:15:22.072958 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.072858 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48487ec7-e8d6-4741-bedc-472a49c37467-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48487ec7-e8d6-4741-bedc-472a49c37467" (UID: "48487ec7-e8d6-4741-bedc-472a49c37467"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:15:22.171988 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.171957 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48487ec7-e8d6-4741-bedc-472a49c37467-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:15:22.171988 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.171984 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzckv\" (UniqueName: \"kubernetes.io/projected/48487ec7-e8d6-4741-bedc-472a49c37467-kube-api-access-lzckv\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:15:22.171988 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.171995 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48487ec7-e8d6-4741-bedc-472a49c37467-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:15:22.494440 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.494411 2560 generic.go:356] "Generic (PLEG): container finished" podID="48487ec7-e8d6-4741-bedc-472a49c37467" containerID="1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16" exitCode=0 Jul 05 14:15:22.494794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.494451 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" Jul 05 14:15:22.494794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.494492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" event={"ID":"48487ec7-e8d6-4741-bedc-472a49c37467","Type":"ContainerDied","Data":"1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16"} Jul 05 14:15:22.494794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.494544 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd" event={"ID":"48487ec7-e8d6-4741-bedc-472a49c37467","Type":"ContainerDied","Data":"c485033fe6d71d6bb976cb21e9b68280eb61f0fffb7350e6f1618c49f048f7ec"} Jul 05 14:15:22.494794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.494572 2560 scope.go:117] "RemoveContainer" containerID="1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16" Jul 05 14:15:22.503758 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.503741 2560 scope.go:117] "RemoveContainer" containerID="1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16" Jul 05 14:15:22.504017 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:15:22.503999 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16\": container with ID starting with 1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16 not found: ID does not exist" containerID="1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16" Jul 05 14:15:22.504073 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.504026 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16"} err="failed to get container status \"1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16\": rpc error: code = NotFound desc = could not find container \"1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16\": container with ID starting with 1ac78ab44f2b90b5c6828b1fff42242ae71fcf6b834ca1aa92d9a554607eaa16 not found: ID does not exist" Jul 05 14:15:22.516403 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.516382 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd"] Jul 05 14:15:22.522115 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:22.522094 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9394f3-76c4c7f8c9-q2rpd"] Jul 05 14:15:23.590659 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:23.590628 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48487ec7-e8d6-4741-bedc-472a49c37467" path="/var/lib/kubelet/pods/48487ec7-e8d6-4741-bedc-472a49c37467/volumes" Jul 05 14:15:27.235421 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:27.235389 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:27.340083 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:27.340055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:27.515691 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:27.515603 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:27.755169 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:27.755126 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:28.072357 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:28.072299 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:28.444559 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:28.444531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:28.580117 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:28.580086 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:31.818535 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.818502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:31.904862 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.904830 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p"] Jul 05 14:15:31.905250 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.905220 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48487ec7-e8d6-4741-bedc-472a49c37467" containerName="maas-api" Jul 05 14:15:31.905250 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.905242 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="48487ec7-e8d6-4741-bedc-472a49c37467" containerName="maas-api" Jul 05 14:15:31.905468 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.905264 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" containerName="maas-api" Jul 05 14:15:31.905468 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.905272 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" containerName="maas-api" Jul 05 14:15:31.905468 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.905335 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8e49f07-1279-46b2-a7dc-ccc1c5c3b5eb" containerName="maas-api" Jul 05 14:15:31.905468 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.905347 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="48487ec7-e8d6-4741-bedc-472a49c37467" containerName="maas-api" Jul 05 14:15:31.909320 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.909288 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:31.912337 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.912322 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-9073be-serving-cert\"" Jul 05 14:15:31.919749 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.919726 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p"] Jul 05 14:15:31.921967 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:31.921946 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:32.018934 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.018907 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:32.051655 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.051623 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbpb5\" (UniqueName: \"kubernetes.io/projected/e1983309-3321-4336-a205-0f2ccc7bc8c8-kube-api-access-mbpb5\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.051815 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.051702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1983309-3321-4336-a205-0f2ccc7bc8c8-maas-api-tls\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.051815 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.051761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1983309-3321-4336-a205-0f2ccc7bc8c8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.132724 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.132692 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:32.152881 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.152855 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbpb5\" (UniqueName: \"kubernetes.io/projected/e1983309-3321-4336-a205-0f2ccc7bc8c8-kube-api-access-mbpb5\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.153005 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.152893 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1983309-3321-4336-a205-0f2ccc7bc8c8-maas-api-tls\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.153005 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.152919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1983309-3321-4336-a205-0f2ccc7bc8c8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.153556 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.153537 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1983309-3321-4336-a205-0f2ccc7bc8c8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.155377 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.155358 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1983309-3321-4336-a205-0f2ccc7bc8c8-maas-api-tls\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.162206 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.162187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbpb5\" (UniqueName: \"kubernetes.io/projected/e1983309-3321-4336-a205-0f2ccc7bc8c8-kube-api-access-mbpb5\") pod \"maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.221013 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.220988 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:32.353552 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.353528 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p"] Jul 05 14:15:32.355509 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:15:32.355474 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1983309_3321_4336_a205_0f2ccc7bc8c8.slice/crio-994bc878182e7da6f6d6d7cf34a479ea90e397e4989eb0d3c7a2755d65e60635 WatchSource:0}: Error finding container 994bc878182e7da6f6d6d7cf34a479ea90e397e4989eb0d3c7a2755d65e60635: Status 404 returned error can't find the container with id 994bc878182e7da6f6d6d7cf34a479ea90e397e4989eb0d3c7a2755d65e60635 Jul 05 14:15:32.497010 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.496937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:32.543383 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.543352 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" event={"ID":"e1983309-3321-4336-a205-0f2ccc7bc8c8","Type":"ContainerStarted","Data":"994bc878182e7da6f6d6d7cf34a479ea90e397e4989eb0d3c7a2755d65e60635"} Jul 05 14:15:32.597789 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:32.597766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:33.549302 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:33.549260 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" event={"ID":"e1983309-3321-4336-a205-0f2ccc7bc8c8","Type":"ContainerStarted","Data":"d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae"} Jul 05 14:15:33.549785 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:33.549385 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:33.572335 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:33.572277 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" podStartSLOduration=2.252687438 podStartE2EDuration="2.57226375s" totalImagesPullingTime="319.576312ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:15:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:15:33.570603484 +0000 UTC m=+2209.750445929" watchObservedRunningTime="2026-07-05 14:15:33.57226375 +0000 UTC m=+2209.752106211" Jul 05 14:15:36.297427 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:36.297389 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:36.623388 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:36.623355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:36.934105 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:36.934029 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:37.047697 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:37.047666 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:39.560905 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:39.560876 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:15:39.913781 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:39.913744 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:39.986778 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:39.986746 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt"] Jul 05 14:15:39.990289 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:39.990268 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:39.993103 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:39.993077 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-a7c515-serving-cert\"" Jul 05 14:15:39.993245 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:39.993219 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:40.001920 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.001894 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt"] Jul 05 14:15:40.014808 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.014778 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.014913 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.014829 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mgt6\" (UniqueName: \"kubernetes.io/projected/b8778d89-9dfd-4772-8230-deb26db81341-kube-api-access-2mgt6\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.014974 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.014911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8778d89-9dfd-4772-8230-deb26db81341-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.070699 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.070669 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:40.115776 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.115741 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.115921 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.115784 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mgt6\" (UniqueName: \"kubernetes.io/projected/b8778d89-9dfd-4772-8230-deb26db81341-kube-api-access-2mgt6\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.115921 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.115811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8778d89-9dfd-4772-8230-deb26db81341-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.115921 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:15:40.115883 2560 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-a7c515-serving-cert: secret "maas-api-e2e-shared-b-a7c515-serving-cert" not found Jul 05 14:15:40.116043 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:15:40.115943 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls podName:b8778d89-9dfd-4772-8230-deb26db81341 nodeName:}" failed. No retries permitted until 2026-07-05 14:15:40.615927052 +0000 UTC m=+2216.795769481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls") pod "maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" (UID: "b8778d89-9dfd-4772-8230-deb26db81341") : secret "maas-api-e2e-shared-b-a7c515-serving-cert" not found Jul 05 14:15:40.116440 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.116424 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8778d89-9dfd-4772-8230-deb26db81341-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.129030 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.129002 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mgt6\" (UniqueName: \"kubernetes.io/projected/b8778d89-9dfd-4772-8230-deb26db81341-kube-api-access-2mgt6\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.341794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.341764 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:40.494135 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.494099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:40.594505 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.594412 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:40.621120 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.621088 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.623459 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.623432 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls\") pod \"maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:40.903611 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:40.903523 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:41.042288 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:41.042263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt"] Jul 05 14:15:41.043579 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:15:41.043553 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8778d89_9dfd_4772_8230_deb26db81341.slice/crio-66ef654ea0b6f07ab2bc5c2c7221e0d1a4d2963a1b3afac0745ed7a24d4dce1d WatchSource:0}: Error finding container 66ef654ea0b6f07ab2bc5c2c7221e0d1a4d2963a1b3afac0745ed7a24d4dce1d: Status 404 returned error can't find the container with id 66ef654ea0b6f07ab2bc5c2c7221e0d1a4d2963a1b3afac0745ed7a24d4dce1d Jul 05 14:15:41.590813 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:41.590786 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:41.590961 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:41.590833 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" event={"ID":"b8778d89-9dfd-4772-8230-deb26db81341","Type":"ContainerStarted","Data":"d12a021de04f3b2e109f166345c50a04dffabcbf59f5ebd243c37012b16200f3"} Jul 05 14:15:41.590961 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:41.590853 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" event={"ID":"b8778d89-9dfd-4772-8230-deb26db81341","Type":"ContainerStarted","Data":"66ef654ea0b6f07ab2bc5c2c7221e0d1a4d2963a1b3afac0745ed7a24d4dce1d"} Jul 05 14:15:41.613012 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:41.612905 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" podStartSLOduration=2.313953887 podStartE2EDuration="2.612888499s" totalImagesPullingTime="298.934612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:15:41.610915918 +0000 UTC m=+2217.790758368" watchObservedRunningTime="2026-07-05 14:15:41.612888499 +0000 UTC m=+2217.792730945" Jul 05 14:15:44.385102 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:44.385069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:47.599930 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:47.599905 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:15:53.330098 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:53.330058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:15:53.430595 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:15:53.430554 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:16:00.305721 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:16:00.305682 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:16:01.233772 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:16:01.233736 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:16:01.300696 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:16:01.300653 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:16:25.023396 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:16:25.022711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:17:49.020695 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:17:49.020656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:17:53.594404 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:17:53.594367 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:18:12.403114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:18:12.403082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:18:13.422338 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:18:13.422297 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:18:37.136356 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:18:37.136292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:18:41.155502 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:18:41.155463 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:18:41.256521 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:18:41.256472 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:18:45.309726 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:18:45.309686 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:18:45.314070 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:18:45.314052 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:19:32.724186 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:19:32.724102 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:20:09.211539 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:09.211505 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:20:09.312322 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:09.312281 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:20:26.392922 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:26.392886 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:20:27.271102 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:27.271063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:20:40.595199 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.595119 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5"] Jul 05 14:20:40.598375 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.598356 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.602510 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.602433 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9073be\"/\"default-dockercfg-bmxrv\"" Jul 05 14:20:40.602510 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.602487 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9073be\"/\"openshift-service-ca.crt\"" Jul 05 14:20:40.602695 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.602437 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9073be\"/\"rate-test-model-9073be-kserve-self-signed-certs\"" Jul 05 14:20:40.602695 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.602586 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9073be\"/\"kube-root-ca.crt\"" Jul 05 14:20:40.618465 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.618439 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5"] Jul 05 14:20:40.633061 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.633035 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:20:40.654581 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.654556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp7lj\" (UniqueName: \"kubernetes.io/projected/015c5025-f393-47da-afc6-469b10234cb2-kube-api-access-cp7lj\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.654688 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.654594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-home\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.654688 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.654616 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-dshm\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.654688 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.654657 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/015c5025-f393-47da-afc6-469b10234cb2-tls-certs\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.654860 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.654748 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-model-cache\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.654860 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.654815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-tmp-dir\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.755787 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.755760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/015c5025-f393-47da-afc6-469b10234cb2-tls-certs\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.755912 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.755810 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-model-cache\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.755980 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.755931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-tmp-dir\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.756093 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.755986 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp7lj\" (UniqueName: \"kubernetes.io/projected/015c5025-f393-47da-afc6-469b10234cb2-kube-api-access-cp7lj\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.756093 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.756013 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-home\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.756093 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.756034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-dshm\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.756252 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.756180 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-model-cache\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.756252 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.756238 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-tmp-dir\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.756430 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.756411 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-home\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.758162 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.758144 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-dshm\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.758293 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.758274 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/015c5025-f393-47da-afc6-469b10234cb2-tls-certs\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.764544 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.764524 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp7lj\" (UniqueName: \"kubernetes.io/projected/015c5025-f393-47da-afc6-469b10234cb2-kube-api-access-cp7lj\") pod \"rate-test-model-9073be-kserve-7f856c7c79-jkhq5\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:40.908634 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:40.908574 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:41.037806 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:41.037778 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5"] Jul 05 14:20:41.039052 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:20:41.039027 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod015c5025_f393_47da_afc6_469b10234cb2.slice/crio-7a74b5791f4ef873477c2147b9d937adebb2c50d4077ab823cea811f26ab0156 WatchSource:0}: Error finding container 7a74b5791f4ef873477c2147b9d937adebb2c50d4077ab823cea811f26ab0156: Status 404 returned error can't find the container with id 7a74b5791f4ef873477c2147b9d937adebb2c50d4077ab823cea811f26ab0156 Jul 05 14:20:41.057732 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:41.057699 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" event={"ID":"015c5025-f393-47da-afc6-469b10234cb2","Type":"ContainerStarted","Data":"7a74b5791f4ef873477c2147b9d937adebb2c50d4077ab823cea811f26ab0156"} Jul 05 14:20:42.064045 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:42.064015 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" event={"ID":"015c5025-f393-47da-afc6-469b10234cb2","Type":"ContainerStarted","Data":"12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0"} Jul 05 14:20:42.082359 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:42.082293 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" podStartSLOduration=2.082278919 podStartE2EDuration="2.082278919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 14:20:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:20:42.081461036 +0000 UTC m=+2518.261303483" watchObservedRunningTime="2026-07-05 14:20:42.082278919 +0000 UTC m=+2518.262121385" Jul 05 14:20:50.908780 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:50.908746 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:50.909176 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:50.908799 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:50.920986 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:50.920964 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:51.116456 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:51.116421 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:20:54.064357 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:54.064301 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:20:54.165937 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:20:54.165907 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:02.219170 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:02.219137 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:02.322908 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:02.322881 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:27.238179 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:27.238141 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:32.265892 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:32.265857 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:32.324619 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:32.324579 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:33.096819 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:33.096782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:45.418671 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:45.418639 2560 scope.go:117] "RemoveContainer" containerID="541a6d3465f02302e643eaf389481ebd4c8d8fe5c850e8967d163e516f9514f0" Jul 05 14:21:48.954722 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:48.954681 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:21:49.021497 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:21:49.021471 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:11.692978 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:11.692902 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5"] Jul 05 14:22:11.717956 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:11.717931 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:12.528531 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:12.528496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:13.598170 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:13.598138 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:14.051749 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.051647 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:14.153215 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.153185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:14.254346 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.254289 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:14.341635 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.341600 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p"] Jul 05 14:22:14.341862 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.341839 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" podUID="e1983309-3321-4336-a205-0f2ccc7bc8c8" containerName="maas-api" containerID="cri-o://d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae" gracePeriod=30 Jul 05 14:22:14.385395 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.385368 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:14.555015 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.554979 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" podUID="e1983309-3321-4336-a205-0f2ccc7bc8c8" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.48:8443/health\": dial tcp 10.134.0.48:8443: connect: connection refused" Jul 05 14:22:14.787062 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.787039 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:22:14.913372 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.913267 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1983309-3321-4336-a205-0f2ccc7bc8c8-openshift-service-ca\") pod \"e1983309-3321-4336-a205-0f2ccc7bc8c8\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " Jul 05 14:22:14.913508 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.913397 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1983309-3321-4336-a205-0f2ccc7bc8c8-maas-api-tls\") pod \"e1983309-3321-4336-a205-0f2ccc7bc8c8\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " Jul 05 14:22:14.913508 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.913430 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbpb5\" (UniqueName: \"kubernetes.io/projected/e1983309-3321-4336-a205-0f2ccc7bc8c8-kube-api-access-mbpb5\") pod \"e1983309-3321-4336-a205-0f2ccc7bc8c8\" (UID: \"e1983309-3321-4336-a205-0f2ccc7bc8c8\") " Jul 05 14:22:14.913649 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.913622 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1983309-3321-4336-a205-0f2ccc7bc8c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1983309-3321-4336-a205-0f2ccc7bc8c8" (UID: "e1983309-3321-4336-a205-0f2ccc7bc8c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:22:14.915440 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.915417 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1983309-3321-4336-a205-0f2ccc7bc8c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1983309-3321-4336-a205-0f2ccc7bc8c8" (UID: "e1983309-3321-4336-a205-0f2ccc7bc8c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:22:14.915515 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:14.915496 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1983309-3321-4336-a205-0f2ccc7bc8c8-kube-api-access-mbpb5" (OuterVolumeSpecName: "kube-api-access-mbpb5") pod "e1983309-3321-4336-a205-0f2ccc7bc8c8" (UID: "e1983309-3321-4336-a205-0f2ccc7bc8c8"). InnerVolumeSpecName "kube-api-access-mbpb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:22:15.014376 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.014351 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1983309-3321-4336-a205-0f2ccc7bc8c8-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:15.014376 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.014372 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbpb5\" (UniqueName: \"kubernetes.io/projected/e1983309-3321-4336-a205-0f2ccc7bc8c8-kube-api-access-mbpb5\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:15.014520 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.014384 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1983309-3321-4336-a205-0f2ccc7bc8c8-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:15.533564 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.533527 2560 generic.go:356] "Generic (PLEG): container finished" podID="e1983309-3321-4336-a205-0f2ccc7bc8c8" containerID="d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae" exitCode=0 Jul 05 14:22:15.533752 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.533572 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" Jul 05 14:22:15.533752 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.533619 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" event={"ID":"e1983309-3321-4336-a205-0f2ccc7bc8c8","Type":"ContainerDied","Data":"d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae"} Jul 05 14:22:15.533752 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.533670 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p" event={"ID":"e1983309-3321-4336-a205-0f2ccc7bc8c8","Type":"ContainerDied","Data":"994bc878182e7da6f6d6d7cf34a479ea90e397e4989eb0d3c7a2755d65e60635"} Jul 05 14:22:15.533752 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.533685 2560 scope.go:117] "RemoveContainer" containerID="d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae" Jul 05 14:22:15.542641 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.542623 2560 scope.go:117] "RemoveContainer" containerID="d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae" Jul 05 14:22:15.542884 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:22:15.542867 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae\": container with ID starting with d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae not found: ID does not exist" containerID="d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae" Jul 05 14:22:15.542945 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.542896 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae"} err="failed to get container status \"d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae\": rpc error: code = NotFound desc = could not find container \"d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae\": container with ID starting with d8f1771a2e4b4c04e4cc014f54622c3a948da37a4f34246785573b74fbb428ae not found: ID does not exist" Jul 05 14:22:15.555951 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.555925 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p"] Jul 05 14:22:15.563759 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.563739 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9073be-847b57f5b9-qr52p"] Jul 05 14:22:15.591400 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:15.591377 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1983309-3321-4336-a205-0f2ccc7bc8c8" path="/var/lib/kubelet/pods/e1983309-3321-4336-a205-0f2ccc7bc8c8/volumes" Jul 05 14:22:19.128192 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:19.128157 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:26.712304 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:26.712250 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" podUID="015c5025-f393-47da-afc6-469b10234cb2" containerName="main" containerID="cri-o://12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0" gracePeriod=45 Jul 05 14:22:26.953183 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:26.953162 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:22:27.107851 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.107825 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/015c5025-f393-47da-afc6-469b10234cb2-tls-certs\") pod \"015c5025-f393-47da-afc6-469b10234cb2\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " Jul 05 14:22:27.107981 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.107887 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-tmp-dir\") pod \"015c5025-f393-47da-afc6-469b10234cb2\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " Jul 05 14:22:27.107981 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.107912 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-dshm\") pod \"015c5025-f393-47da-afc6-469b10234cb2\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " Jul 05 14:22:27.107981 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.107934 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-model-cache\") pod \"015c5025-f393-47da-afc6-469b10234cb2\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " Jul 05 14:22:27.108089 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.107982 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-home\") pod \"015c5025-f393-47da-afc6-469b10234cb2\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " Jul 05 14:22:27.108089 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.108031 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp7lj\" (UniqueName: \"kubernetes.io/projected/015c5025-f393-47da-afc6-469b10234cb2-kube-api-access-cp7lj\") pod \"015c5025-f393-47da-afc6-469b10234cb2\" (UID: \"015c5025-f393-47da-afc6-469b10234cb2\") " Jul 05 14:22:27.108176 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.108100 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "015c5025-f393-47da-afc6-469b10234cb2" (UID: "015c5025-f393-47da-afc6-469b10234cb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:22:27.108216 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.108198 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "015c5025-f393-47da-afc6-469b10234cb2" (UID: "015c5025-f393-47da-afc6-469b10234cb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:22:27.108278 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.108249 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-home" (OuterVolumeSpecName: "home") pod "015c5025-f393-47da-afc6-469b10234cb2" (UID: "015c5025-f393-47da-afc6-469b10234cb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:22:27.108351 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.108299 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-tmp-dir\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:27.108351 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.108339 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-model-cache\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:27.110031 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.110000 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-dshm" (OuterVolumeSpecName: "dshm") pod "015c5025-f393-47da-afc6-469b10234cb2" (UID: "015c5025-f393-47da-afc6-469b10234cb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:22:27.110131 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.110048 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/015c5025-f393-47da-afc6-469b10234cb2-kube-api-access-cp7lj" (OuterVolumeSpecName: "kube-api-access-cp7lj") pod "015c5025-f393-47da-afc6-469b10234cb2" (UID: "015c5025-f393-47da-afc6-469b10234cb2"). InnerVolumeSpecName "kube-api-access-cp7lj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:22:27.110131 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.110052 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/015c5025-f393-47da-afc6-469b10234cb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "015c5025-f393-47da-afc6-469b10234cb2" (UID: "015c5025-f393-47da-afc6-469b10234cb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:22:27.209607 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.209586 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-dshm\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:27.209607 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.209607 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/015c5025-f393-47da-afc6-469b10234cb2-home\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:27.209738 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.209617 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp7lj\" (UniqueName: \"kubernetes.io/projected/015c5025-f393-47da-afc6-469b10234cb2-kube-api-access-cp7lj\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:27.209738 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.209628 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/015c5025-f393-47da-afc6-469b10234cb2-tls-certs\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:27.593914 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.593882 2560 generic.go:356] "Generic (PLEG): container finished" podID="015c5025-f393-47da-afc6-469b10234cb2" containerID="12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0" exitCode=0 Jul 05 14:22:27.594068 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.593940 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" Jul 05 14:22:27.594068 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.593956 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" event={"ID":"015c5025-f393-47da-afc6-469b10234cb2","Type":"ContainerDied","Data":"12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0"} Jul 05 14:22:27.594068 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.593984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5" event={"ID":"015c5025-f393-47da-afc6-469b10234cb2","Type":"ContainerDied","Data":"7a74b5791f4ef873477c2147b9d937adebb2c50d4077ab823cea811f26ab0156"} Jul 05 14:22:27.594068 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.594000 2560 scope.go:117] "RemoveContainer" containerID="12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0" Jul 05 14:22:27.608218 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.608200 2560 scope.go:117] "RemoveContainer" containerID="12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0" Jul 05 14:22:27.608484 ip-10-0-142-97 kubenswrapper[2560]: E0705 14:22:27.608463 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0\": container with ID starting with 12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0 not found: ID does not exist" containerID="12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0" Jul 05 14:22:27.608537 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.608493 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0"} err="failed to get container status \"12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0\": rpc error: code = NotFound desc = could not find container \"12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0\": container with ID starting with 12a2f8c6362dfb458b532a161533d87efb9c7e3c3dc7b5b8b3faa83ccac673f0 not found: ID does not exist" Jul 05 14:22:27.617401 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.617377 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5"] Jul 05 14:22:27.623028 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:27.623007 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9073be/rate-test-model-9073be-kserve-7f856c7c79-jkhq5"] Jul 05 14:22:29.590742 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:29.590710 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="015c5025-f393-47da-afc6-469b10234cb2" path="/var/lib/kubelet/pods/015c5025-f393-47da-afc6-469b10234cb2/volumes" Jul 05 14:22:56.226511 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:56.226481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:56.518236 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:56.518150 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:56.733473 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:56.733435 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:56.902765 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:56.902734 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:57.290193 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.290109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:57.390021 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.389986 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:57.490578 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.490547 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:57.561192 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.561098 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt"] Jul 05 14:22:57.561391 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.561358 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" podUID="b8778d89-9dfd-4772-8230-deb26db81341" containerName="maas-api" containerID="cri-o://d12a021de04f3b2e109f166345c50a04dffabcbf59f5ebd243c37012b16200f3" gracePeriod=30 Jul 05 14:22:57.584113 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.584090 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:22:57.594591 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.594558 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" podUID="b8778d89-9dfd-4772-8230-deb26db81341" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.49:8443/health\": dial tcp 10.134.0.49:8443: connect: connection refused" Jul 05 14:22:57.741986 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.741959 2560 generic.go:356] "Generic (PLEG): container finished" podID="b8778d89-9dfd-4772-8230-deb26db81341" containerID="d12a021de04f3b2e109f166345c50a04dffabcbf59f5ebd243c37012b16200f3" exitCode=0 Jul 05 14:22:57.742116 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.742040 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" event={"ID":"b8778d89-9dfd-4772-8230-deb26db81341","Type":"ContainerDied","Data":"d12a021de04f3b2e109f166345c50a04dffabcbf59f5ebd243c37012b16200f3"} Jul 05 14:22:57.804652 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.804632 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:22:57.837492 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.837458 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls\") pod \"b8778d89-9dfd-4772-8230-deb26db81341\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " Jul 05 14:22:57.837595 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.837508 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8778d89-9dfd-4772-8230-deb26db81341-openshift-service-ca\") pod \"b8778d89-9dfd-4772-8230-deb26db81341\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " Jul 05 14:22:57.837595 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.837551 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mgt6\" (UniqueName: \"kubernetes.io/projected/b8778d89-9dfd-4772-8230-deb26db81341-kube-api-access-2mgt6\") pod \"b8778d89-9dfd-4772-8230-deb26db81341\" (UID: \"b8778d89-9dfd-4772-8230-deb26db81341\") " Jul 05 14:22:57.837838 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.837816 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8778d89-9dfd-4772-8230-deb26db81341-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8778d89-9dfd-4772-8230-deb26db81341" (UID: "b8778d89-9dfd-4772-8230-deb26db81341"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:22:57.839382 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.839360 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8778d89-9dfd-4772-8230-deb26db81341" (UID: "b8778d89-9dfd-4772-8230-deb26db81341"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 14:22:57.839487 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.839469 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8778d89-9dfd-4772-8230-deb26db81341-kube-api-access-2mgt6" (OuterVolumeSpecName: "kube-api-access-2mgt6") pod "b8778d89-9dfd-4772-8230-deb26db81341" (UID: "b8778d89-9dfd-4772-8230-deb26db81341"). InnerVolumeSpecName "kube-api-access-2mgt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:22:57.938227 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.938203 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8778d89-9dfd-4772-8230-deb26db81341-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:57.938227 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.938225 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8778d89-9dfd-4772-8230-deb26db81341-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:57.938388 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:57.938234 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mgt6\" (UniqueName: \"kubernetes.io/projected/b8778d89-9dfd-4772-8230-deb26db81341-kube-api-access-2mgt6\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 05 14:22:58.748120 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:58.748089 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" Jul 05 14:22:58.748573 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:58.748120 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt" event={"ID":"b8778d89-9dfd-4772-8230-deb26db81341","Type":"ContainerDied","Data":"66ef654ea0b6f07ab2bc5c2c7221e0d1a4d2963a1b3afac0745ed7a24d4dce1d"} Jul 05 14:22:58.748573 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:58.748177 2560 scope.go:117] "RemoveContainer" containerID="d12a021de04f3b2e109f166345c50a04dffabcbf59f5ebd243c37012b16200f3" Jul 05 14:22:58.771359 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:58.771334 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt"] Jul 05 14:22:58.779204 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:58.779180 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a7c515-779c6b9676-gdrlt"] Jul 05 14:22:59.590568 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:22:59.590532 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8778d89-9dfd-4772-8230-deb26db81341" path="/var/lib/kubelet/pods/b8778d89-9dfd-4772-8230-deb26db81341/volumes" Jul 05 14:23:03.070070 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:03.070038 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:23:03.130769 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:03.130737 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:23:03.341620 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:03.341586 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:23:03.571557 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:03.571527 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:23:08.732503 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:08.732464 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5b59cd5649-b68dz_70a26def-7569-4331-a895-742379fdeeed/maas-api/0.log" Jul 05 14:23:09.063571 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:09.063487 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-696f68cd7-v5nnq_51a79347-9247-46d7-b27d-e6826d9c947e/manager/0.log" Jul 05 14:23:09.407224 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:09.407198 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f79c67cfc-cpgtg_e49baf1a-1120-4185-931f-084ef9a6ea75/copy-manifests/0.log" Jul 05 14:23:09.425572 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:09.425548 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f79c67cfc-cpgtg_e49baf1a-1120-4185-931f-084ef9a6ea75/manager/0.log" Jul 05 14:23:09.792731 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:09.792655 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-q2dt7_daa74441-440d-4e47-92b7-360c72f653ff/postgres/0.log" Jul 05 14:23:10.566888 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.566858 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st_296dc6e5-d09d-46ae-ade5-8f448bd240d6/util/0.log" Jul 05 14:23:10.583164 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.583135 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st_296dc6e5-d09d-46ae-ade5-8f448bd240d6/pull/0.log" Jul 05 14:23:10.595947 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.595916 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st_296dc6e5-d09d-46ae-ade5-8f448bd240d6/extract/0.log" Jul 05 14:23:10.704503 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.704479 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn_28c22f75-7e29-42ea-8a78-032e925944fe/util/0.log" Jul 05 14:23:10.710894 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.710872 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn_28c22f75-7e29-42ea-8a78-032e925944fe/pull/0.log" Jul 05 14:23:10.717154 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.717131 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn_28c22f75-7e29-42ea-8a78-032e925944fe/extract/0.log" Jul 05 14:23:10.828832 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.828811 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv_b883cab1-a89b-435e-b330-68119697e3d1/util/0.log" Jul 05 14:23:10.835418 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.835398 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv_b883cab1-a89b-435e-b330-68119697e3d1/pull/0.log" Jul 05 14:23:10.841816 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.841800 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv_b883cab1-a89b-435e-b330-68119697e3d1/extract/0.log" Jul 05 14:23:10.957796 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.957771 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h_2eec340b-e114-4fdd-b435-7e118b909439/util/0.log" Jul 05 14:23:10.964963 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.964940 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h_2eec340b-e114-4fdd-b435-7e118b909439/pull/0.log" Jul 05 14:23:10.971630 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:10.971605 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h_2eec340b-e114-4fdd-b435-7e118b909439/extract/0.log" Jul 05 14:23:12.361541 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:12.361510 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wvvb_3e2c46a8-542d-46d0-afae-d39717b167f6/discovery/0.log" Jul 05 14:23:12.943389 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:12.943360 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-q7r8k_e1c17b93-b079-4979-895f-c4c90fadb951/payload-processing/0.log" Jul 05 14:23:13.402451 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:13.402423 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-cgpmk_61b93bc5-971e-4b93-9630-bbb9f8193275/main/0.log" Jul 05 14:23:13.849007 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:13.848975 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-j7s2p_f365a76e-c6ce-4d57-a86b-a50b64e16382/main/0.log" Jul 05 14:23:18.251164 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:18.251134 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:23:18.327689 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:18.327641 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" limit=1024 Jul 05 14:23:21.039784 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:21.039731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9wxg4_4a740b28-33ae-4e0f-b746-6e5f6cd5eeec/global-pull-secret-syncer/0.log" Jul 05 14:23:21.131761 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:21.131731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nmbbx_a36113f7-8e41-48ce-9a66-156e196e4c60/konnectivity-agent/0.log" Jul 05 14:23:21.266576 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:21.266544 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-97.ec2.internal_f2f4209627eb6491cbb2535de8e653eb/haproxy/0.log" Jul 05 14:23:25.240921 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.240888 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st_296dc6e5-d09d-46ae-ade5-8f448bd240d6/extract/0.log" Jul 05 14:23:25.258302 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.258278 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st_296dc6e5-d09d-46ae-ade5-8f448bd240d6/util/0.log" Jul 05 14:23:25.276763 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.276729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nq5st_296dc6e5-d09d-46ae-ade5-8f448bd240d6/pull/0.log" Jul 05 14:23:25.303068 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.303044 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn_28c22f75-7e29-42ea-8a78-032e925944fe/extract/0.log" Jul 05 14:23:25.322720 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.322699 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn_28c22f75-7e29-42ea-8a78-032e925944fe/util/0.log" Jul 05 14:23:25.342602 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.342582 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ws6kn_28c22f75-7e29-42ea-8a78-032e925944fe/pull/0.log" Jul 05 14:23:25.368843 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.368823 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv_b883cab1-a89b-435e-b330-68119697e3d1/extract/0.log" Jul 05 14:23:25.393375 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.393356 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv_b883cab1-a89b-435e-b330-68119697e3d1/util/0.log" Jul 05 14:23:25.411243 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.411226 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jdhrv_b883cab1-a89b-435e-b330-68119697e3d1/pull/0.log" Jul 05 14:23:25.435915 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.435900 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h_2eec340b-e114-4fdd-b435-7e118b909439/extract/0.log" Jul 05 14:23:25.453420 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.453402 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h_2eec340b-e114-4fdd-b435-7e118b909439/util/0.log" Jul 05 14:23:25.474696 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:25.474681 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w678h_2eec340b-e114-4fdd-b435-7e118b909439/pull/0.log" Jul 05 14:23:27.825292 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:27.825262 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4/alertmanager/0.log" Jul 05 14:23:27.848467 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:27.848443 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4/config-reloader/0.log" Jul 05 14:23:27.875938 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:27.875916 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4/kube-rbac-proxy-web/0.log" Jul 05 14:23:27.898461 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:27.898445 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4/kube-rbac-proxy/0.log" Jul 05 14:23:27.921936 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:27.921915 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4/kube-rbac-proxy-metric/0.log" Jul 05 14:23:27.946922 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:27.946907 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4/prom-label-proxy/0.log" Jul 05 14:23:27.967742 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:27.967722 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ad9669eb-cece-4d2f-ae16-5e24c4a4d4c4/init-config-reloader/0.log" Jul 05 14:23:28.110919 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:28.110828 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d6b77c859-zcqsj_bd1b475b-07a5-40b3-ba48-ac526830b507/metrics-server/0.log" Jul 05 14:23:28.177012 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:28.176982 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gmzwk_fbe060a4-5912-4815-89f3-a18b6ea064c7/node-exporter/0.log" Jul 05 14:23:28.206669 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:28.206650 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gmzwk_fbe060a4-5912-4815-89f3-a18b6ea064c7/kube-rbac-proxy/0.log" Jul 05 14:23:28.227624 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:28.227605 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gmzwk_fbe060a4-5912-4815-89f3-a18b6ea064c7/init-textfile/0.log" Jul 05 14:23:31.030380 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030344 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq"] Jul 05 14:23:31.030794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030695 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1983309-3321-4336-a205-0f2ccc7bc8c8" containerName="maas-api" Jul 05 14:23:31.030794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030708 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1983309-3321-4336-a205-0f2ccc7bc8c8" containerName="maas-api" Jul 05 14:23:31.030794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030717 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8778d89-9dfd-4772-8230-deb26db81341" containerName="maas-api" Jul 05 14:23:31.030794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030723 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8778d89-9dfd-4772-8230-deb26db81341" containerName="maas-api" Jul 05 14:23:31.030794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030749 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="015c5025-f393-47da-afc6-469b10234cb2" containerName="main" Jul 05 14:23:31.030794 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030755 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="015c5025-f393-47da-afc6-469b10234cb2" containerName="main" Jul 05 14:23:31.031111 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030810 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8778d89-9dfd-4772-8230-deb26db81341" containerName="maas-api" Jul 05 14:23:31.031111 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030816 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1983309-3321-4336-a205-0f2ccc7bc8c8" containerName="maas-api" Jul 05 14:23:31.031111 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.030822 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="015c5025-f393-47da-afc6-469b10234cb2" containerName="main" Jul 05 14:23:31.034114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.034095 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.036753 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.036726 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-489kn\"/\"openshift-service-ca.crt\"" Jul 05 14:23:31.036864 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.036827 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-489kn\"/\"kube-root-ca.crt\"" Jul 05 14:23:31.036864 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.036839 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-489kn\"/\"default-dockercfg-2cnkl\"" Jul 05 14:23:31.042030 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.042007 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq"] Jul 05 14:23:31.194505 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.194477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-proc\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.194657 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.194515 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-sys\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.194657 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.194545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-lib-modules\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.194657 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.194602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-podres\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.194657 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.194646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mlkp\" (UniqueName: \"kubernetes.io/projected/21865c70-30ad-46bf-a24b-0cd01dfe4b80-kube-api-access-6mlkp\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.295930 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.295872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-lib-modules\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.295930 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.295899 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-podres\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.295930 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.295924 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mlkp\" (UniqueName: \"kubernetes.io/projected/21865c70-30ad-46bf-a24b-0cd01dfe4b80-kube-api-access-6mlkp\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.296114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.295958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-proc\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.296114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.295989 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-sys\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.296114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.296034 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-lib-modules\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.296114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.296041 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-podres\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.296114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.296049 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-sys\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.296114 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.296086 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21865c70-30ad-46bf-a24b-0cd01dfe4b80-proc\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.305401 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.305383 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mlkp\" (UniqueName: \"kubernetes.io/projected/21865c70-30ad-46bf-a24b-0cd01dfe4b80-kube-api-access-6mlkp\") pod \"perf-node-gather-daemonset-r8vqq\" (UID: \"21865c70-30ad-46bf-a24b-0cd01dfe4b80\") " pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.344425 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.344407 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.474828 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.474802 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq"] Jul 05 14:23:31.475660 ip-10-0-142-97 kubenswrapper[2560]: W0705 14:23:31.475634 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod21865c70_30ad_46bf_a24b_0cd01dfe4b80.slice/crio-13dd4c126caa60424a635a20dc098625468e51440eaf8177eed5f9664f800666 WatchSource:0}: Error finding container 13dd4c126caa60424a635a20dc098625468e51440eaf8177eed5f9664f800666: Status 404 returned error can't find the container with id 13dd4c126caa60424a635a20dc098625468e51440eaf8177eed5f9664f800666 Jul 05 14:23:31.906922 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.906884 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" event={"ID":"21865c70-30ad-46bf-a24b-0cd01dfe4b80","Type":"ContainerStarted","Data":"502c911f72829a7e4b8f6afd0038e9686b9df3aecb4fa493a8ab0422e15ae99c"} Jul 05 14:23:31.907103 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.906942 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" event={"ID":"21865c70-30ad-46bf-a24b-0cd01dfe4b80","Type":"ContainerStarted","Data":"13dd4c126caa60424a635a20dc098625468e51440eaf8177eed5f9664f800666"} Jul 05 14:23:31.907103 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.906996 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:31.924461 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:31.924417 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" podStartSLOduration=0.924404703 podStartE2EDuration="924.404703ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 14:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:23:31.922382243 +0000 UTC m=+2688.102224754" watchObservedRunningTime="2026-07-05 14:23:31.924404703 +0000 UTC m=+2688.104247149" Jul 05 14:23:33.140393 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:33.140363 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-77lql_3908baf2-7deb-42c1-8ccb-71a7a3008b0d/dns/0.log" Jul 05 14:23:33.160263 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:33.160242 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-77lql_3908baf2-7deb-42c1-8ccb-71a7a3008b0d/kube-rbac-proxy/0.log" Jul 05 14:23:33.247288 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:33.247254 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r7c9v_d1f48763-233d-4180-9ae8-cd2e5934ef3b/dns-node-resolver/0.log" Jul 05 14:23:33.881152 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:33.881128 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6lmkh_43d2952f-50ec-4ce3-9239-0d5d69d5ded0/node-ca/0.log" Jul 05 14:23:35.058727 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:35.058692 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wvvb_3e2c46a8-542d-46d0-afae-d39717b167f6/discovery/0.log" Jul 05 14:23:35.266486 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:35.266456 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-q7r8k_e1c17b93-b079-4979-895f-c4c90fadb951/payload-processing/0.log" Jul 05 14:23:35.986691 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:35.986666 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hgq7h_8a5d4105-f02a-4be3-aafa-e7edb24c1163/serve-healthcheck-canary/0.log" Jul 05 14:23:36.490421 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:36.490389 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2svds_58ebfbf3-7b88-4db4-ba06-6a6c69d388ea/kube-rbac-proxy/0.log" Jul 05 14:23:36.509536 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:36.509508 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2svds_58ebfbf3-7b88-4db4-ba06-6a6c69d388ea/exporter/0.log" Jul 05 14:23:36.529091 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:36.529073 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2svds_58ebfbf3-7b88-4db4-ba06-6a6c69d388ea/extractor/0.log" Jul 05 14:23:37.921598 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:37.921571 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-489kn/perf-node-gather-daemonset-r8vqq" Jul 05 14:23:39.052441 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:39.052410 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5b59cd5649-b68dz_70a26def-7569-4331-a895-742379fdeeed/maas-api/0.log" Jul 05 14:23:39.250465 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:39.250432 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-696f68cd7-v5nnq_51a79347-9247-46d7-b27d-e6826d9c947e/manager/0.log" Jul 05 14:23:39.492542 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:39.492506 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f79c67cfc-cpgtg_e49baf1a-1120-4185-931f-084ef9a6ea75/manager/0.log" Jul 05 14:23:39.508275 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:39.508250 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f79c67cfc-cpgtg_e49baf1a-1120-4185-931f-084ef9a6ea75/copy-manifests/0.log" Jul 05 14:23:39.623881 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:39.623851 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-q2dt7_daa74441-440d-4e47-92b7-360c72f653ff/postgres/0.log" Jul 05 14:23:40.965582 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:40.965504 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bf64756c4-pk87j_13394b7d-9ebc-4d76-9dd4-f5309e87d9c2/manager/0.log" Jul 05 14:23:41.161224 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:41.161190 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wvvb_3e2c46a8-542d-46d0-afae-d39717b167f6/discovery/0.log" Jul 05 14:23:41.348493 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:41.348463 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-q7r8k_e1c17b93-b079-4979-895f-c4c90fadb951/payload-processing/0.log" Jul 05 14:23:42.859731 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:42.859700 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6wvvb_3e2c46a8-542d-46d0-afae-d39717b167f6/discovery/0.log" Jul 05 14:23:43.040670 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:43.040645 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-q7r8k_e1c17b93-b079-4979-895f-c4c90fadb951/payload-processing/0.log" Jul 05 14:23:45.346874 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:45.346844 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:23:45.351390 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:45.351367 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:23:47.618381 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.618354 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4ct4_5730bfca-1a7a-449b-8205-5d938dd443ef/kube-multus-additional-cni-plugins/0.log" Jul 05 14:23:47.639674 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.639647 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4ct4_5730bfca-1a7a-449b-8205-5d938dd443ef/egress-router-binary-copy/0.log" Jul 05 14:23:47.660616 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.660584 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4ct4_5730bfca-1a7a-449b-8205-5d938dd443ef/cni-plugins/0.log" Jul 05 14:23:47.678890 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.678862 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4ct4_5730bfca-1a7a-449b-8205-5d938dd443ef/bond-cni-plugin/0.log" Jul 05 14:23:47.697178 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.697150 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4ct4_5730bfca-1a7a-449b-8205-5d938dd443ef/routeoverride-cni/0.log" Jul 05 14:23:47.717065 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.717033 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4ct4_5730bfca-1a7a-449b-8205-5d938dd443ef/whereabouts-cni-bincopy/0.log" Jul 05 14:23:47.738058 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.738029 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4ct4_5730bfca-1a7a-449b-8205-5d938dd443ef/whereabouts-cni/0.log" Jul 05 14:23:47.807302 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.807272 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-srgqx_c57b80fc-d60a-4a29-9101-c71955868288/kube-multus/0.log" Jul 05 14:23:47.930137 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.930068 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n66xq_c424c3fa-89b7-43e4-80e9-a09466ee2d89/network-metrics-daemon/0.log" Jul 05 14:23:47.947318 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:47.947293 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n66xq_c424c3fa-89b7-43e4-80e9-a09466ee2d89/kube-rbac-proxy/0.log" Jul 05 14:23:48.990941 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:48.990909 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-controller/0.log" Jul 05 14:23:49.006398 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.006369 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/1.log" Jul 05 14:23:49.006550 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.006473 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovn-acl-logging/0.log" Jul 05 14:23:49.029032 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.029009 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/kube-rbac-proxy-node/0.log" Jul 05 14:23:49.054442 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.054406 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/kube-rbac-proxy-ovn-metrics/0.log" Jul 05 14:23:49.075493 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.075460 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/northd/0.log" Jul 05 14:23:49.100269 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.100242 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/nbdb/0.log" Jul 05 14:23:49.123978 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.123947 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/sbdb/0.log" Jul 05 14:23:49.205221 ip-10-0-142-97 kubenswrapper[2560]: I0705 14:23:49.205191 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dq5h4_9759e393-3424-4efd-afac-56eedc206f49/ovnkube-controller/0.log"