Jul 18 11:20:04.997241 ip-10-0-142-41 systemd[1]: Starting Kubernetes Kubelet... Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.353550 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358233 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358252 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358256 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358259 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358262 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358265 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358268 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:20:05.409832 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358271 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358274 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358277 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358281 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358283 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358286 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358288 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358291 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358294 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358297 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358301 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358304 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358307 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358310 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358318 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358321 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358324 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358327 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358329 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358332 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:20:05.411264 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358335 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358337 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358341 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358344 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358348 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358353 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358356 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358358 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358361 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358363 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358366 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358369 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358372 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358374 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358377 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358380 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358383 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358385 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358388 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:20:05.412692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358391 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358394 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358396 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358399 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358401 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358404 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358406 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358409 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358414 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358418 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358420 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358424 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358427 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358430 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358433 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358436 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358439 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358443 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358445 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:20:05.413526 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358448 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358451 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358453 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358456 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358459 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358461 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358464 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358488 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358492 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358496 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358499 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358504 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358507 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358510 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358514 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358517 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358520 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358523 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358526 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:20:05.414282 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358529 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358532 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358535 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358538 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358540 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358543 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358546 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358548 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358551 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358553 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358557 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358560 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358563 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358566 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.358568 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359329 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359338 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359341 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359344 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359347 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359350 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:20:05.414899 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359353 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359355 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359358 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359361 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359363 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359366 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359368 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359372 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359375 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359378 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359380 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359384 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359387 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359390 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359392 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359395 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359398 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359400 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359403 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:20:05.415556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359406 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359409 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359412 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359415 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359417 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359420 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359423 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359426 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359429 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359432 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359434 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359439 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359443 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359446 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359449 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359451 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359454 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359456 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359459 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359461 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:20:05.416225 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359464 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359467 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359469 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359472 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359475 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359477 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359481 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359484 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359487 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359490 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359493 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359495 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359498 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359501 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359504 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359507 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359509 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359512 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359515 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:20:05.417032 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359518 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359520 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359523 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359526 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359528 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359532 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359535 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359538 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359541 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359544 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359547 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359549 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359552 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359554 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359557 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359560 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359563 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359566 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359568 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359571 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:20:05.417640 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359573 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359576 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359579 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359583 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359586 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359594 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359597 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359600 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359602 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359605 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359608 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359610 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359613 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359616 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.359618 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360045 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360051 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360054 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360058 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:20:05.418391 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360060 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360063 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360066 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360069 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360071 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360074 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360077 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360079 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360083 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360086 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360088 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360096 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360099 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360102 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360104 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360108 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360112 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360115 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360119 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360122 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:20:05.419052 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360125 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360127 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360130 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360133 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360135 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360138 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360141 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360143 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360146 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360148 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360151 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360158 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360160 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360163 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360165 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360168 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360171 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360173 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360176 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360179 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:20:05.419616 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360182 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360185 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360188 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360190 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360192 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360195 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360197 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360200 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360202 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360206 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360215 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360218 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360220 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360223 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360226 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360228 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360231 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360234 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360236 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360239 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:20:05.420224 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360241 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360244 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360246 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360250 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360252 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360255 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360257 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360260 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360263 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360265 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360268 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360270 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360274 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360276 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360279 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360282 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360284 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360287 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360289 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360291 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:20:05.420752 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360294 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360297 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360301 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360304 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360307 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360311 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360314 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360317 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360319 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360323 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360325 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360328 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360331 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360333 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.360336 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360422 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360431 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360438 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360442 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360447 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360450 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 11:20:05.421331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360458 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360464 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360467 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360470 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360474 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360477 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360482 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360485 2585 flags.go:64] FLAG: --cgroup-root="" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360488 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360491 2585 flags.go:64] FLAG: --client-ca-file="" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360494 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360497 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360501 2585 flags.go:64] FLAG: --cluster-domain="" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360505 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360509 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360512 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360515 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360519 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360523 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360526 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360529 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360532 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360535 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360539 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 11:20:05.422288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360542 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360545 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360550 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360554 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360556 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360559 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360562 2585 flags.go:64] FLAG: --enable-server="true" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360566 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360575 2585 flags.go:64] FLAG: --event-burst="100" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360578 2585 flags.go:64] FLAG: --event-qps="50" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360581 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360584 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360587 2585 flags.go:64] FLAG: --eviction-hard="" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360591 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360594 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360598 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360601 2585 flags.go:64] FLAG: --eviction-soft="" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360604 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360607 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360609 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360612 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360617 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360620 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360622 2585 flags.go:64] FLAG: --feature-gates="" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360626 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 11:20:05.423472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360629 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360632 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360636 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360639 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360642 2585 flags.go:64] FLAG: --help="false" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360645 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360648 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360651 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360654 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360659 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360662 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360665 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360668 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360671 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360674 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360676 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360679 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360682 2585 flags.go:64] FLAG: --kube-reserved="" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360685 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360688 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360691 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360694 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360697 2585 flags.go:64] FLAG: --lock-file="" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360700 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 11:20:05.424775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360703 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360706 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360712 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360714 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360719 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360722 2585 flags.go:64] FLAG: --logging-format="text" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360725 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360728 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360731 2585 flags.go:64] FLAG: --manifest-url="" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360733 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360738 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360741 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360745 2585 flags.go:64] FLAG: --max-pods="110" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360748 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360751 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360754 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360757 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360763 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360766 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360769 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360777 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360779 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360783 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 11:20:05.425609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360786 2585 flags.go:64] FLAG: --pod-cidr="" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360789 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360794 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360797 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360800 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360803 2585 flags.go:64] FLAG: --port="10250" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360806 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360809 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09fdda061ed6841e7" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360812 2585 flags.go:64] FLAG: --qos-reserved="" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360815 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360818 2585 flags.go:64] FLAG: --register-node="true" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360821 2585 flags.go:64] FLAG: --register-with-taints="" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360825 2585 flags.go:64] FLAG: --registry-burst="10" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360830 2585 flags.go:64] FLAG: --registry-qps="5" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360833 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360836 2585 flags.go:64] FLAG: --reserved-memory="" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360839 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360842 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360845 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360848 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360851 2585 flags.go:64] FLAG: --runonce="false" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360854 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360857 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360860 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360863 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 11:20:05.428317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360866 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360869 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360874 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360877 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360893 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360896 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360899 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360902 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360906 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360908 2585 flags.go:64] FLAG: --system-cgroups="" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360911 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360917 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360921 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360924 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360928 2585 flags.go:64] FLAG: --tls-min-version="" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360931 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360934 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360937 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360940 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360943 2585 flags.go:64] FLAG: --v="2" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360950 2585 flags.go:64] FLAG: --version="false" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360954 2585 flags.go:64] FLAG: --vmodule="" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360958 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.360961 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361086 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:20:05.429152 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361090 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361093 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361096 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361098 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361101 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361104 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361106 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361109 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361112 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361116 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361119 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361122 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361124 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361127 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361130 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361132 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361135 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361138 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361140 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:20:05.429983 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361143 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361146 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361149 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361151 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361154 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361157 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361159 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361162 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361166 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361168 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361171 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361173 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361176 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361178 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361181 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361184 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361186 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361189 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361191 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361194 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:20:05.431236 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361196 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361198 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361202 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361205 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361208 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361210 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361213 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361216 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361218 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361221 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361223 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361225 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361228 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361231 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361233 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361236 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361238 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361241 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361243 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361246 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:20:05.431875 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361250 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361252 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361255 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361257 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361260 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361262 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361266 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361270 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361273 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361277 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361280 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361283 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361287 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361290 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361294 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361297 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361300 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361303 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361305 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:20:05.432503 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361309 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361312 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361316 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361319 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361321 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361324 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361327 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361329 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361332 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361334 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361337 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361339 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361342 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361346 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361348 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361351 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361353 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361356 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361359 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:20:05.433067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.361361 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.361936 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.385659 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.385683 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.385700 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.385705 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385771 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385778 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385782 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385787 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385791 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385794 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385798 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385802 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:20:05.433696 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385805 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385808 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385811 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385814 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385817 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385820 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385823 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385826 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385829 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385831 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385834 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385837 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385841 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385843 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385851 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385855 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385859 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385864 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385867 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:20:05.434252 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385870 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385872 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385875 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385878 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385899 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385902 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385905 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385908 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385911 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385914 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385917 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385919 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385922 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385924 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385927 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385930 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385932 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385935 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385938 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385941 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:20:05.434798 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385943 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385946 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385948 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385951 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385954 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385957 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385960 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385964 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385967 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385969 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385973 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385976 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385979 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385981 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385983 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385986 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385989 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385991 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385994 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385996 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:20:05.435546 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.385999 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386002 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386004 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386008 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386010 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386013 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386015 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386018 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386020 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386023 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386025 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386028 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386030 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386033 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386035 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386038 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386040 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386043 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386045 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386050 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:20:05.436151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386053 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386056 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386060 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386063 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386065 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386068 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386070 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386073 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386076 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386079 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386081 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386084 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.386089 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386236 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386242 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:20:05.506543 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386245 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:20:05.443144 ip-10-0-142-41 systemd[1]: Started Kubernetes Kubelet. Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386248 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386251 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386254 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386257 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386260 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386263 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386265 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386268 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386271 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386273 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386275 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386279 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386282 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386284 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386287 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386289 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386294 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386296 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386300 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:20:05.507274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386303 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386306 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386310 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386313 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386316 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386319 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386321 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386324 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386327 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386330 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386333 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386336 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386338 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386341 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386343 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386347 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386350 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386353 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386356 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:20:05.508093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386358 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386361 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386363 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386366 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386369 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386372 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386375 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386378 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386380 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386383 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386387 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386391 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386395 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386398 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386400 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386403 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386406 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386408 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386411 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386413 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:20:05.508788 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386416 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386419 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386421 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386424 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386427 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386429 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386432 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386434 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386437 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386440 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386443 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386445 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386448 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386450 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386453 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386455 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386458 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386460 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386463 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386465 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:20:05.509569 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386468 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386470 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386474 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386476 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386481 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386483 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386486 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386489 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386491 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386494 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386496 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386499 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386502 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386504 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386507 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386510 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386512 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:20:05.510313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:05.386515 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.386519 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.386653 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.388823 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.389977 2585 server.go:1019] "Starting client certificate rotation" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.390079 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.390120 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.392025 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.397085 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.409271 2585 log.go:25] "Validated CRI v1 runtime API" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.415955 2585 log.go:25] "Validated CRI v1 image API" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.416827 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 11:20:05.510965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.418978 2585 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 94dbafa8-9961-4afe-a42c-7b1803cfe382:/dev/nvme0n1p3 9b422b15-772f-41e6-bcf8-8a61cdbe0be6:/dev/nvme0n1p4] Jul 18 11:20:05.511421 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.418995 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 11:20:05.511421 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.419871 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.424112 2585 manager.go:216] Machine: {Timestamp:2026-07-18 11:20:05.423351948 +0000 UTC m=+0.325116077 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099431 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec225ea45fb1fa07f6f2f3606aa75ff2 SystemUUID:ec225ea4-5fb1-fa07-f6f2-f3606aa75ff2 BootID:8efb2d11-07d0-4585-9f44-cd50f4e3f584 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:72:3f:2a:d2:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:72:3f:2a:d2:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:74:77:3a:22:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.424262 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.424450 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.426867 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.426922 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-41.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.427498 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.427510 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.427519 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.427532 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.428685 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.432570 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.432710 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.432748 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.432766 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.432778 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.432788 2585 kubelet.go:398] "Adding apiserver pod source" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.432796 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 11:20:05.511823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.433325 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.433347 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.435172 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.436651 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.436685 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437753 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437767 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437776 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437784 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437794 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437800 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437808 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437813 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437838 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437844 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437858 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.437864 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.438656 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.438667 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.441914 2585 server.go:1268] "Started kubelet" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.442064 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.442095 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.442197 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.443212 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.447409 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.447571 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-skcqj" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.447586 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-skcqj" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.453056 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.454587 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.454592 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.455297 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.455308 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.454832 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462084 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462480 2585 factory.go:55] Registering systemd factory Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462509 2585 factory.go:223] Registration of the systemd container factory successfully Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462548 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462559 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462774 2585 factory.go:153] Registering CRI-O factory Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462801 2585 factory.go:223] Registration of the crio container factory successfully Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462875 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462920 2585 factory.go:103] Registering Raw factory Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.462935 2585 manager.go:1196] Started watching for new ooms in manager Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.463431 2585 manager.go:349] Starting recovery of all containers Jul 18 11:20:05.512507 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.464062 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.464188 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-41.ec2.internal.18c35e5712f1f7e9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-41.ec2.internal,UID:ip-10-0-142-41.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-41.ec2.internal,},FirstTimestamp:2026-07-18 11:20:05.441861609 +0000 UTC m=+0.343625738,LastTimestamp:2026-07-18 11:20:05.441861609 +0000 UTC m=+0.343625738,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-41.ec2.internal,}" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.465042 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-41.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.465134 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.465271 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.465290 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.471989 2585 manager.go:354] Recovery completed Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.477872 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.479785 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.479827 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.479842 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.480451 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.480464 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.480482 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.480455 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.481240 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.481265 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.481274 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.485257 2585 policy_none.go:49] "None policy: Start" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.485273 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.485285 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 11:20:05.513870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.487862 2585 policy_none.go:47] "Start" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.518641 2585 manager.go:337] "Starting Device Plugin manager" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.518768 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.518780 2585 server.go:87] "Starting device plugin registration server" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.519107 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.519120 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.519377 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.519447 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.519454 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.520010 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.520059 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.606073 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.607458 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.607510 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.607542 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.607587 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.613659 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.619410 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.620372 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.620407 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.620422 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.620448 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.627770 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.627799 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-41.ec2.internal\": node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:05.667634 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.641154 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.708222 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal"] Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.708311 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.710146 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.710178 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.710190 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.712530 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.712647 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.712680 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.713498 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.713524 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.713535 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.713501 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.713593 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.713603 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.716722 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.716753 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.717621 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.717644 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.717659 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.741454 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.747055 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-41.ec2.internal\" not found" node="ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.752936 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-41.ec2.internal\" not found" node="ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.763665 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac67fe649f5ad8c00d895f8f51798ec7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal\" (UID: \"ac67fe649f5ad8c00d895f8f51798ec7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.763699 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac67fe649f5ad8c00d895f8f51798ec7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal\" (UID: \"ac67fe649f5ad8c00d895f8f51798ec7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.763717 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5cb379f3a8937535961aac401e8f7558-config\") pod \"kube-apiserver-proxy-ip-10-0-142-41.ec2.internal\" (UID: \"5cb379f3a8937535961aac401e8f7558\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.842604 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.863831 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac67fe649f5ad8c00d895f8f51798ec7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal\" (UID: \"ac67fe649f5ad8c00d895f8f51798ec7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.863881 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac67fe649f5ad8c00d895f8f51798ec7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal\" (UID: \"ac67fe649f5ad8c00d895f8f51798ec7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.876625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.863919 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5cb379f3a8937535961aac401e8f7558-config\") pod \"kube-apiserver-proxy-ip-10-0-142-41.ec2.internal\" (UID: \"5cb379f3a8937535961aac401e8f7558\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.880337 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.863960 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5cb379f3a8937535961aac401e8f7558-config\") pod \"kube-apiserver-proxy-ip-10-0-142-41.ec2.internal\" (UID: \"5cb379f3a8937535961aac401e8f7558\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.880337 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.863947 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac67fe649f5ad8c00d895f8f51798ec7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal\" (UID: \"ac67fe649f5ad8c00d895f8f51798ec7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:05.880337 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:05.864004 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac67fe649f5ad8c00d895f8f51798ec7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal\" (UID: \"ac67fe649f5ad8c00d895f8f51798ec7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:06.052371 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:05.943316 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:06.052371 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:06.050043 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:06.052371 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.050053 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" Jul 18 11:20:06.140031 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.078977 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:06.242645 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:06.150816 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:06.251339 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:06.251304 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:06.353102 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:06.353013 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:06.390001 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.389965 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 11:20:06.390186 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.390168 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 11:20:06.454311 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:06.454257 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:06.463318 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.463271 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 11:15:06 +0000 UTC" deadline="2028-04-25 14:42:26.419983746 +0000 UTC" Jul 18 11:20:06.463318 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.463311 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15531h22m19.956676562s" Jul 18 11:20:06.554478 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:06.554433 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-41.ec2.internal\" not found" Jul 18 11:20:06.580026 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.579986 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:20:06.655371 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.655334 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" Jul 18 11:20:06.672452 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.672291 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 11:20:06.673849 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.673819 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" Jul 18 11:20:06.681416 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.681395 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 11:20:06.754054 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:06.754014 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb379f3a8937535961aac401e8f7558.slice/crio-b13a432dd5341136446f705c66f4a87aa2579f2e278fa99e1eb650e48a64e091 WatchSource:0}: Error finding container b13a432dd5341136446f705c66f4a87aa2579f2e278fa99e1eb650e48a64e091: Status 404 returned error can't find the container with id b13a432dd5341136446f705c66f4a87aa2579f2e278fa99e1eb650e48a64e091 Jul 18 11:20:06.754452 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:06.754437 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac67fe649f5ad8c00d895f8f51798ec7.slice/crio-88a85c811c758d0f680106c576f7aedb7a00ec805ef5bba4e6d9c22f484237bb WatchSource:0}: Error finding container 88a85c811c758d0f680106c576f7aedb7a00ec805ef5bba4e6d9c22f484237bb: Status 404 returned error can't find the container with id 88a85c811c758d0f680106c576f7aedb7a00ec805ef5bba4e6d9c22f484237bb Jul 18 11:20:06.760254 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.760236 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:20:06.872185 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:06.872091 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:20:07.053059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.053020 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:20:07.434348 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.434043 2585 apiserver.go:52] "Watching apiserver" Jul 18 11:20:07.445995 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.443660 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 11:20:07.446324 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.446289 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal","openshift-multus/multus-additional-cni-plugins-2bbjc","openshift-multus/network-metrics-daemon-bx2rv","kube-system/konnectivity-agent-gx6rn","openshift-image-registry/node-ca-6tf8c","openshift-multus/multus-4k74j","openshift-network-diagnostics/network-check-target-trnrr","openshift-network-operator/iptables-alerter-smc2t","openshift-ovn-kubernetes/ovnkube-node-6q55b","kube-system/global-pull-secret-syncer-8jxp7","kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n","openshift-cluster-node-tuning-operator/tuned-84dgn","openshift-dns/node-resolver-d8qgd"] Jul 18 11:20:07.449285 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.449251 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:07.449442 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.449347 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:07.452829 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.452512 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.454977 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.454934 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:07.455557 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.455034 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:07.455557 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.455529 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 11:20:07.456899 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.456854 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 11:20:07.457203 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.457185 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6j56s\"" Jul 18 11:20:07.457439 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.457419 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:20:07.457511 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.457466 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.459594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.459571 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.460343 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.460319 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 11:20:07.460440 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.460318 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 11:20:07.460440 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.460333 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rgcdx\"" Jul 18 11:20:07.462007 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.461981 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.462626 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.462599 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 11:20:07.462821 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.462800 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 11:20:07.462821 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.462818 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vxztn\"" Jul 18 11:20:07.464079 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.463299 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 11:20:07.464655 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.464635 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.465316 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.465293 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 11:20:07.465987 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.465605 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-km8qn\"" Jul 18 11:20:07.465987 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.465798 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 11:20:07.465987 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.465843 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 11:20:07.465987 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.465906 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 11:20:07.469827 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.469149 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 11:20:07.469827 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.469394 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.469827 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.469419 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 11:20:07.469827 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.469460 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 11:20:07.471058 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.470624 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 11:20:07.471058 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.470633 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 11:20:07.471058 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.470682 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6wgth\"" Jul 18 11:20:07.471058 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.470697 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.471445 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.471143 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 11:20:07.472180 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.472060 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 11:20:07.472915 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.472834 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.473019 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.472928 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:07.473261 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.473217 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 11:20:07.473346 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.473292 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 11:20:07.473757 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.473629 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ztfqj\"" Jul 18 11:20:07.473848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.473801 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p985n\"" Jul 18 11:20:07.474575 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.474221 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 11:20:07.474575 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.474230 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 11:20:07.475523 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.475504 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.479086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.478251 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 11:20:07.479086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.478419 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.480643 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.480096 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 11:20:07.480643 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.480164 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hd8fs\"" Jul 18 11:20:07.480643 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.480097 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:20:07.481692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.481316 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 11:20:07.481692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.481546 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 11:20:07.482731 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.482548 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-krv7n\"" Jul 18 11:20:07.482731 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.482641 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 11:20:07.489036 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.488725 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-cni-netd\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489036 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.488787 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-kubernetes\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.489036 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.488829 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-socket-dir-parent\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.489036 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.488900 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-kubelet\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489036 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.488958 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-slash\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489036 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489025 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-run-netns\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489052 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-ovn\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489080 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-var-lib-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489188 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-lib-modules\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.489427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489225 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:07.489427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489291 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-multus-certs\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.489427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489366 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxs79\" (UniqueName: \"kubernetes.io/projected/22dac10b-2945-4cd3-9acc-5a5a2a40c808-kube-api-access-sxs79\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489424 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-os-release\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.489761 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489487 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-k8s-cni-cncf-io\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.489761 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489516 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-cni-bin\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.489761 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489574 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovnkube-config\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.489913 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489840 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-host\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.490006 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.489986 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/072e83ea-7610-4224-9ce9-bb0da55af877-tmp\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490160 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-cni-bin\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490369 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-run\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490402 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-hostroot\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490464 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-daemon-config\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490742 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysctl-d\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490784 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnmh\" (UniqueName: \"kubernetes.io/projected/072e83ea-7610-4224-9ce9-bb0da55af877-kube-api-access-cvnmh\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490816 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovn-node-metrics-cert\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.490898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490867 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovnkube-script-lib\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.490958 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491023 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-modprobe-d\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491151 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-systemd-units\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491195 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-systemd\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491225 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-env-overrides\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491301 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491344 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-log-socket\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.491365 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491364 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-run-ovn-kubernetes\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.491768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491472 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-var-lib-kubelet\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.491768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491516 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/072e83ea-7610-4224-9ce9-bb0da55af877-etc-tuned\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.491768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491574 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b7c0627-1350-4093-8768-8dae93d211b9-host\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.491768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491623 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-conf-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.491768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491679 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69157e6f-b546-4043-897f-e914056249c5-konnectivity-ca\") pod \"konnectivity-agent-gx6rn\" (UID: \"69157e6f-b546-4043-897f-e914056249c5\") " pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.491768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491748 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kvh8\" (UniqueName: \"kubernetes.io/projected/0b7c0627-1350-4093-8768-8dae93d211b9-kube-api-access-9kvh8\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.492088 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491859 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-system-cni-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492088 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491900 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-cni-multus\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492088 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.491933 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-etc-kubernetes\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492088 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492036 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgjg6\" (UniqueName: \"kubernetes.io/projected/4af2d7f8-d600-497e-b76f-6853bf602faa-kube-api-access-sgjg6\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492091 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.492297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492130 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdlm5\" (UniqueName: \"kubernetes.io/projected/ab36abb8-bb6e-458f-b909-277a53c37126-kube-api-access-bdlm5\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:07.492297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492168 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-cni-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492228 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-sys\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.492297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492280 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b7c0627-1350-4093-8768-8dae93d211b9-serviceca\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.492553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492312 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-systemd\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.492553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492375 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-cnibin\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492402 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4af2d7f8-d600-497e-b76f-6853bf602faa-cni-binary-copy\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-kubelet\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-etc-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.492553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492520 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysconfig\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.492828 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492659 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysctl-conf\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.492828 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492704 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69157e6f-b546-4043-897f-e914056249c5-agent-certs\") pod \"konnectivity-agent-gx6rn\" (UID: \"69157e6f-b546-4043-897f-e914056249c5\") " pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.492828 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492775 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-netns\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.492996 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.492831 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-node-log\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.495496 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.493215 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-txld8" Jul 18 11:20:07.498765 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.498740 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-txld8" Jul 18 11:20:07.563877 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.563844 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 11:20:07.593102 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593059 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-cni-bin\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.593300 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593112 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovnkube-config\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.593300 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593192 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-cni-bin\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.593300 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-host\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593303 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/072e83ea-7610-4224-9ce9-bb0da55af877-tmp\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593369 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-cni-bin\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593387 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-host\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593412 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-run\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593438 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-hostroot\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5502dd71-8ce2-4279-91ed-211ab06df23f-host-slash\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593509 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-device-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.593548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593534 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-os-release\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593559 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-daemon-config\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593584 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysctl-d\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593613 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnmh\" (UniqueName: \"kubernetes.io/projected/072e83ea-7610-4224-9ce9-bb0da55af877-kube-api-access-cvnmh\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593638 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovn-node-metrics-cert\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593666 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovnkube-script-lib\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593697 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-cni-bin\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593693 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-modprobe-d\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593779 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-systemd-units\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593758 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593804 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-systemd\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593827 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-env-overrides\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593866 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d58rx\" (UniqueName: \"kubernetes.io/projected/6a96c874-8c04-41a0-bf56-16613a1a69c0-kube-api-access-d58rx\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593865 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-run\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593945 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.593861 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-hostroot\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.594019 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594003 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovnkube-config\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594043 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-systemd\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594198 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-modprobe-d\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594243 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-systemd-units\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594416 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-log-socket\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594501 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-run-ovn-kubernetes\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594526 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovnkube-script-lib\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594548 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-var-lib-kubelet\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594553 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-daemon-config\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594614 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/072e83ea-7610-4224-9ce9-bb0da55af877-etc-tuned\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594615 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-log-socket\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-run-ovn-kubernetes\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594650 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b7c0627-1350-4093-8768-8dae93d211b9-host\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594659 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysctl-d\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594679 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/22dac10b-2945-4cd3-9acc-5a5a2a40c808-env-overrides\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594684 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a96c874-8c04-41a0-bf56-16613a1a69c0-hosts-file\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594689 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-var-lib-kubelet\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.594926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594711 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-conf-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594755 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69157e6f-b546-4043-897f-e914056249c5-konnectivity-ca\") pod \"konnectivity-agent-gx6rn\" (UID: \"69157e6f-b546-4043-897f-e914056249c5\") " pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594747 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-conf-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594713 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b7c0627-1350-4093-8768-8dae93d211b9-host\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594827 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kvh8\" (UniqueName: \"kubernetes.io/projected/0b7c0627-1350-4093-8768-8dae93d211b9-kube-api-access-9kvh8\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.594855 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74vqr\" (UniqueName: \"kubernetes.io/projected/4a5682f8-5600-4843-b384-5c4e7189dbef-kube-api-access-74vqr\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595028 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a96c874-8c04-41a0-bf56-16613a1a69c0-tmp-dir\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595061 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-system-cni-dir\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595085 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1109b33-d650-4822-b3ab-08d1cd14d5a4-kubelet-config\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595119 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1109b33-d650-4822-b3ab-08d1cd14d5a4-dbus\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595149 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-system-cni-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595170 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-cni-multus\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595199 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-etc-kubernetes\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595219 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-system-cni-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595226 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgjg6\" (UniqueName: \"kubernetes.io/projected/4af2d7f8-d600-497e-b76f-6853bf602faa-kube-api-access-sgjg6\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595242 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-cni-multus\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595268 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.595771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595301 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-etc-kubernetes\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595305 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdlm5\" (UniqueName: \"kubernetes.io/projected/ab36abb8-bb6e-458f-b909-277a53c37126-kube-api-access-bdlm5\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595343 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595416 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-cni-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595422 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69157e6f-b546-4043-897f-e914056249c5-konnectivity-ca\") pod \"konnectivity-agent-gx6rn\" (UID: \"69157e6f-b546-4043-897f-e914056249c5\") " pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595441 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-sys\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595477 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b7c0627-1350-4093-8768-8dae93d211b9-serviceca\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-registration-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595538 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-systemd\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595505 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-sys\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595579 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-cni-dir\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595582 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595585 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-systemd\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-cnibin\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595651 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4af2d7f8-d600-497e-b76f-6853bf602faa-cni-binary-copy\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.596671 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595663 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-cnibin\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-kubelet\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595699 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-etc-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595725 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysconfig\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595770 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysctl-conf\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595771 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-var-lib-kubelet\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69157e6f-b546-4043-897f-e914056249c5-agent-certs\") pod \"konnectivity-agent-gx6rn\" (UID: \"69157e6f-b546-4043-897f-e914056249c5\") " pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595838 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvfzd\" (UniqueName: \"kubernetes.io/projected/5502dd71-8ce2-4279-91ed-211ab06df23f-kube-api-access-kvfzd\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595888 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-netns\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595955 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-node-log\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.595986 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-cni-netd\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596012 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-kubernetes\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596025 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-netns\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596041 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-socket-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596069 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-etc-selinux\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596096 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-sys-fs\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596099 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysconfig\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.597521 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596142 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-cnibin\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596150 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-node-log\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596195 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-etc-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-socket-dir-parent\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596232 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-kubelet\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596243 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-multus-socket-dir-parent\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596277 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4af2d7f8-d600-497e-b76f-6853bf602faa-cni-binary-copy\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596292 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-slash\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596341 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-run-netns\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596361 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-cni-netd\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596368 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-ovn\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596395 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-sysctl-conf\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596397 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596409 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-etc-kubernetes\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-slash\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596474 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-kubelet\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596500 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b7c0627-1350-4093-8768-8dae93d211b9-serviceca\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596513 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-var-lib-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.598382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596526 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-run-ovn\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596542 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-lib-modules\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596591 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-var-lib-openvswitch\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596596 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596627 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5502dd71-8ce2-4279-91ed-211ab06df23f-iptables-alerter-script\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596632 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596673 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/072e83ea-7610-4224-9ce9-bb0da55af877-lib-modules\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-multus-certs\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596719 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22dac10b-2945-4cd3-9acc-5a5a2a40c808-host-run-netns\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596815 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-multus-certs\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.596867 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596890 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxs79\" (UniqueName: \"kubernetes.io/projected/22dac10b-2945-4cd3-9acc-5a5a2a40c808-kube-api-access-sxs79\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.596938 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs podName:ab36abb8-bb6e-458f-b909-277a53c37126 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:08.096917047 +0000 UTC m=+2.998681188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs") pod "network-metrics-daemon-bx2rv" (UID: "ab36abb8-bb6e-458f-b909-277a53c37126") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.596979 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.597019 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x696s\" (UniqueName: \"kubernetes.io/projected/38ac6113-5bbc-4894-a994-2436d51392e3-kube-api-access-x696s\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.597051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-os-release\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.597078 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-k8s-cni-cncf-io\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.599093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.597224 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-os-release\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.599812 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.597265 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4af2d7f8-d600-497e-b76f-6853bf602faa-host-run-k8s-cni-cncf-io\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.599812 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.599296 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/072e83ea-7610-4224-9ce9-bb0da55af877-etc-tuned\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.599927 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.599848 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/072e83ea-7610-4224-9ce9-bb0da55af877-tmp\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.600286 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.600251 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/22dac10b-2945-4cd3-9acc-5a5a2a40c808-ovn-node-metrics-cert\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.600480 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.600398 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69157e6f-b546-4043-897f-e914056249c5-agent-certs\") pod \"konnectivity-agent-gx6rn\" (UID: \"69157e6f-b546-4043-897f-e914056249c5\") " pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.601460 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.601428 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:20:07.601460 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.601459 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:20:07.601460 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.601476 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rk7xd for pod openshift-network-diagnostics/network-check-target-trnrr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:07.601460 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.601543 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd podName:bb148327-e17d-470a-98ac-3790b9196f32 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:08.101522785 +0000 UTC m=+3.003286928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rk7xd" (UniqueName: "kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd") pod "network-check-target-trnrr" (UID: "bb148327-e17d-470a-98ac-3790b9196f32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:07.604608 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.604222 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kvh8\" (UniqueName: \"kubernetes.io/projected/0b7c0627-1350-4093-8768-8dae93d211b9-kube-api-access-9kvh8\") pod \"node-ca-6tf8c\" (UID: \"0b7c0627-1350-4093-8768-8dae93d211b9\") " pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.605767 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.605735 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgjg6\" (UniqueName: \"kubernetes.io/projected/4af2d7f8-d600-497e-b76f-6853bf602faa-kube-api-access-sgjg6\") pod \"multus-4k74j\" (UID: \"4af2d7f8-d600-497e-b76f-6853bf602faa\") " pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.605911 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.605889 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnmh\" (UniqueName: \"kubernetes.io/projected/072e83ea-7610-4224-9ce9-bb0da55af877-kube-api-access-cvnmh\") pod \"tuned-84dgn\" (UID: \"072e83ea-7610-4224-9ce9-bb0da55af877\") " pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.606950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.606914 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxs79\" (UniqueName: \"kubernetes.io/projected/22dac10b-2945-4cd3-9acc-5a5a2a40c808-kube-api-access-sxs79\") pod \"ovnkube-node-6q55b\" (UID: \"22dac10b-2945-4cd3-9acc-5a5a2a40c808\") " pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.606950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.606958 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdlm5\" (UniqueName: \"kubernetes.io/projected/ab36abb8-bb6e-458f-b909-277a53c37126-kube-api-access-bdlm5\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:07.616067 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.616006 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" event={"ID":"ac67fe649f5ad8c00d895f8f51798ec7","Type":"ContainerStarted","Data":"88a85c811c758d0f680106c576f7aedb7a00ec805ef5bba4e6d9c22f484237bb"} Jul 18 11:20:07.617534 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.617501 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" event={"ID":"5cb379f3a8937535961aac401e8f7558","Type":"ContainerStarted","Data":"b13a432dd5341136446f705c66f4a87aa2579f2e278fa99e1eb650e48a64e091"} Jul 18 11:20:07.698368 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698286 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.698368 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698330 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5502dd71-8ce2-4279-91ed-211ab06df23f-host-slash\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.698368 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698350 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-device-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.698368 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698368 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-os-release\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.698692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698444 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5502dd71-8ce2-4279-91ed-211ab06df23f-host-slash\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.698692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698466 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-device-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.698692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698456 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-os-release\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.698692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d58rx\" (UniqueName: \"kubernetes.io/projected/6a96c874-8c04-41a0-bf56-16613a1a69c0-kube-api-access-d58rx\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.698692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a96c874-8c04-41a0-bf56-16613a1a69c0-hosts-file\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.698692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698650 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74vqr\" (UniqueName: \"kubernetes.io/projected/4a5682f8-5600-4843-b384-5c4e7189dbef-kube-api-access-74vqr\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.698692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698680 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a96c874-8c04-41a0-bf56-16613a1a69c0-tmp-dir\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698725 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a96c874-8c04-41a0-bf56-16613a1a69c0-hosts-file\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698874 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-system-cni-dir\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698931 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-system-cni-dir\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698945 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1109b33-d650-4822-b3ab-08d1cd14d5a4-kubelet-config\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698968 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1109b33-d650-4822-b3ab-08d1cd14d5a4-dbus\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699005 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699017 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a96c874-8c04-41a0-bf56-16613a1a69c0-tmp-dir\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699027 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699026 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1109b33-d650-4822-b3ab-08d1cd14d5a4-dbus\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.698977 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1109b33-d650-4822-b3ab-08d1cd14d5a4-kubelet-config\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699053 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-registration-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699095 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-registration-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699112 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.699132 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:07.699136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699120 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699226 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvfzd\" (UniqueName: \"kubernetes.io/projected/5502dd71-8ce2-4279-91ed-211ab06df23f-kube-api-access-kvfzd\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:07.699245 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret podName:d1109b33-d650-4822-b3ab-08d1cd14d5a4 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:08.199223995 +0000 UTC m=+3.100988116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret") pod "global-pull-secret-syncer-8jxp7" (UID: "d1109b33-d650-4822-b3ab-08d1cd14d5a4") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-socket-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699284 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699309 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-etc-selinux\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699335 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-sys-fs\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699358 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-cnibin\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699371 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-socket-dir\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699390 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699430 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5502dd71-8ce2-4279-91ed-211ab06df23f-iptables-alerter-script\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699436 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-sys-fs\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699458 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699486 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x696s\" (UniqueName: \"kubernetes.io/projected/38ac6113-5bbc-4894-a994-2436d51392e3-kube-api-access-x696s\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699502 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a5682f8-5600-4843-b384-5c4e7189dbef-etc-selinux\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.699926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.699560 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38ac6113-5bbc-4894-a994-2436d51392e3-cnibin\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.700653 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.700007 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.700653 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.700056 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38ac6113-5bbc-4894-a994-2436d51392e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.700653 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.700087 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5502dd71-8ce2-4279-91ed-211ab06df23f-iptables-alerter-script\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.708548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.708483 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74vqr\" (UniqueName: \"kubernetes.io/projected/4a5682f8-5600-4843-b384-5c4e7189dbef-kube-api-access-74vqr\") pod \"aws-ebs-csi-driver-node-jpr8n\" (UID: \"4a5682f8-5600-4843-b384-5c4e7189dbef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:07.709024 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.708997 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x696s\" (UniqueName: \"kubernetes.io/projected/38ac6113-5bbc-4894-a994-2436d51392e3-kube-api-access-x696s\") pod \"multus-additional-cni-plugins-2bbjc\" (UID: \"38ac6113-5bbc-4894-a994-2436d51392e3\") " pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.709819 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.709781 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d58rx\" (UniqueName: \"kubernetes.io/projected/6a96c874-8c04-41a0-bf56-16613a1a69c0-kube-api-access-d58rx\") pod \"node-resolver-d8qgd\" (UID: \"6a96c874-8c04-41a0-bf56-16613a1a69c0\") " pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.709916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.709829 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvfzd\" (UniqueName: \"kubernetes.io/projected/5502dd71-8ce2-4279-91ed-211ab06df23f-kube-api-access-kvfzd\") pod \"iptables-alerter-smc2t\" (UID: \"5502dd71-8ce2-4279-91ed-211ab06df23f\") " pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.771115 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.771075 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-84dgn" Jul 18 11:20:07.779083 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.779052 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:07.799065 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.799025 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6tf8c" Jul 18 11:20:07.806417 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.806380 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k74j" Jul 18 11:20:07.815750 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.815715 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:07.823930 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.823876 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d8qgd" Jul 18 11:20:07.832072 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.832038 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" Jul 18 11:20:07.843261 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.843224 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-smc2t" Jul 18 11:20:07.848298 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:07.848275 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" Jul 18 11:20:08.103025 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:08.102990 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:08.103242 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:08.103082 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:08.103242 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.103196 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:20:08.103242 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.103230 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:20:08.103387 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.103246 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rk7xd for pod openshift-network-diagnostics/network-check-target-trnrr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:08.103387 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.103201 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:08.103387 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.103316 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd podName:bb148327-e17d-470a-98ac-3790b9196f32 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:09.103295542 +0000 UTC m=+4.005059681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rk7xd" (UniqueName: "kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd") pod "network-check-target-trnrr" (UID: "bb148327-e17d-470a-98ac-3790b9196f32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:08.103387 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.103348 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs podName:ab36abb8-bb6e-458f-b909-277a53c37126 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:09.103330736 +0000 UTC m=+4.005094853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs") pod "network-metrics-daemon-bx2rv" (UID: "ab36abb8-bb6e-458f-b909-277a53c37126") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:08.203675 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:08.203621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:08.203853 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.203776 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:08.203933 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:08.203857 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret podName:d1109b33-d650-4822-b3ab-08d1cd14d5a4 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:09.203834735 +0000 UTC m=+4.105598856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret") pod "global-pull-secret-syncer-8jxp7" (UID: "d1109b33-d650-4822-b3ab-08d1cd14d5a4") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:08.502181 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:08.502140 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 11:15:08 +0000 UTC" deadline="2028-01-28 20:53:13.656415061 +0000 UTC" Jul 18 11:20:08.502181 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:08.502179 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13425h33m5.154239095s" Jul 18 11:20:08.737692 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.737660 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22dac10b_2945_4cd3_9acc_5a5a2a40c808.slice/crio-58f91f075777b795c6308f9a5d06c9b23f1aa291f5d61a6fe1346cf460c286a6 WatchSource:0}: Error finding container 58f91f075777b795c6308f9a5d06c9b23f1aa291f5d61a6fe1346cf460c286a6: Status 404 returned error can't find the container with id 58f91f075777b795c6308f9a5d06c9b23f1aa291f5d61a6fe1346cf460c286a6 Jul 18 11:20:08.739313 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.739184 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5682f8_5600_4843_b384_5c4e7189dbef.slice/crio-8315e2923ced721543ee7438894087f1e5370e023f820b71eed254099eef3cc6 WatchSource:0}: Error finding container 8315e2923ced721543ee7438894087f1e5370e023f820b71eed254099eef3cc6: Status 404 returned error can't find the container with id 8315e2923ced721543ee7438894087f1e5370e023f820b71eed254099eef3cc6 Jul 18 11:20:08.743490 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.743462 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072e83ea_7610_4224_9ce9_bb0da55af877.slice/crio-cefb027486ff65ad0cdf55383571041d0e167fe667015c44accdc33a16dc92ca WatchSource:0}: Error finding container cefb027486ff65ad0cdf55383571041d0e167fe667015c44accdc33a16dc92ca: Status 404 returned error can't find the container with id cefb027486ff65ad0cdf55383571041d0e167fe667015c44accdc33a16dc92ca Jul 18 11:20:08.746913 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.746826 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a96c874_8c04_41a0_bf56_16613a1a69c0.slice/crio-072cbe33241748ad5015cadbd1f03e80a2cd1983ff9979ffd637dac0844b09c3 WatchSource:0}: Error finding container 072cbe33241748ad5015cadbd1f03e80a2cd1983ff9979ffd637dac0844b09c3: Status 404 returned error can't find the container with id 072cbe33241748ad5015cadbd1f03e80a2cd1983ff9979ffd637dac0844b09c3 Jul 18 11:20:08.748510 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.748475 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af2d7f8_d600_497e_b76f_6853bf602faa.slice/crio-ec5af7264ac3ec94331fe6aaace7961cb202d62a903463fbb7f153f760ac9de1 WatchSource:0}: Error finding container ec5af7264ac3ec94331fe6aaace7961cb202d62a903463fbb7f153f760ac9de1: Status 404 returned error can't find the container with id ec5af7264ac3ec94331fe6aaace7961cb202d62a903463fbb7f153f760ac9de1 Jul 18 11:20:08.774274 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.774238 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5502dd71_8ce2_4279_91ed_211ab06df23f.slice/crio-05c0617d4c26099e0a61d3932b055e995a0c17f776520f53361ea1d7dec4b8c1 WatchSource:0}: Error finding container 05c0617d4c26099e0a61d3932b055e995a0c17f776520f53361ea1d7dec4b8c1: Status 404 returned error can't find the container with id 05c0617d4c26099e0a61d3932b055e995a0c17f776520f53361ea1d7dec4b8c1 Jul 18 11:20:08.775352 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.775324 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69157e6f_b546_4043_897f_e914056249c5.slice/crio-1523916f267d06673067ff301c6d51fc69914ae77cb2503d0c12e4535c61c48a WatchSource:0}: Error finding container 1523916f267d06673067ff301c6d51fc69914ae77cb2503d0c12e4535c61c48a: Status 404 returned error can't find the container with id 1523916f267d06673067ff301c6d51fc69914ae77cb2503d0c12e4535c61c48a Jul 18 11:20:08.776151 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.776079 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7c0627_1350_4093_8768_8dae93d211b9.slice/crio-724b6a60dbca7e1a25ef8ddd407c9624efbaaff5323eafb87cd38143c2c3cf27 WatchSource:0}: Error finding container 724b6a60dbca7e1a25ef8ddd407c9624efbaaff5323eafb87cd38143c2c3cf27: Status 404 returned error can't find the container with id 724b6a60dbca7e1a25ef8ddd407c9624efbaaff5323eafb87cd38143c2c3cf27 Jul 18 11:20:08.776872 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:08.776836 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ac6113_5bbc_4894_a994_2436d51392e3.slice/crio-f77f7f171b5e57965685275f1915e241a94ff8d8919b26849d9cc37be83cd1b4 WatchSource:0}: Error finding container f77f7f171b5e57965685275f1915e241a94ff8d8919b26849d9cc37be83cd1b4: Status 404 returned error can't find the container with id f77f7f171b5e57965685275f1915e241a94ff8d8919b26849d9cc37be83cd1b4 Jul 18 11:20:09.112616 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.112272 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:09.112616 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.112561 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:09.112616 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.112501 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:09.112871 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.112685 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs podName:ab36abb8-bb6e-458f-b909-277a53c37126 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:11.112664354 +0000 UTC m=+6.014428473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs") pod "network-metrics-daemon-bx2rv" (UID: "ab36abb8-bb6e-458f-b909-277a53c37126") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:09.112871 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.112741 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:20:09.112871 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.112782 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:20:09.112871 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.112795 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rk7xd for pod openshift-network-diagnostics/network-check-target-trnrr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:09.112871 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.112859 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd podName:bb148327-e17d-470a-98ac-3790b9196f32 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:11.112842697 +0000 UTC m=+6.014606818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rk7xd" (UniqueName: "kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd") pod "network-check-target-trnrr" (UID: "bb148327-e17d-470a-98ac-3790b9196f32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:09.213666 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.213627 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:09.213870 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.213854 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:09.213981 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.213958 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret podName:d1109b33-d650-4822-b3ab-08d1cd14d5a4 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:11.213919693 +0000 UTC m=+6.115683811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret") pod "global-pull-secret-syncer-8jxp7" (UID: "d1109b33-d650-4822-b3ab-08d1cd14d5a4") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:09.503200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.503085 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 11:15:08 +0000 UTC" deadline="2028-01-25 14:01:12.635569857 +0000 UTC" Jul 18 11:20:09.503200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.503163 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13346h41m3.132412882s" Jul 18 11:20:09.608917 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.608328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:09.608917 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.608385 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:09.608917 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.608329 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:09.608917 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.608479 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:09.608917 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.608599 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:09.608917 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:09.608696 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:09.645284 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.645238 2585 generic.go:356] "Generic (PLEG): container finished" podID="ac67fe649f5ad8c00d895f8f51798ec7" containerID="df1c7f8733be31c9d4696d4f36effeaaa5f6c0efa332e2cc0d1787790899d21d" exitCode=0 Jul 18 11:20:09.645713 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.645637 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" event={"ID":"ac67fe649f5ad8c00d895f8f51798ec7","Type":"ContainerDied","Data":"df1c7f8733be31c9d4696d4f36effeaaa5f6c0efa332e2cc0d1787790899d21d"} Jul 18 11:20:09.669080 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.669028 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" event={"ID":"5cb379f3a8937535961aac401e8f7558","Type":"ContainerStarted","Data":"303ce2457b79b9b7cbf63130f8b41037047904179d2f03b6ebcf829099d4140d"} Jul 18 11:20:09.677179 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.677138 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerStarted","Data":"f77f7f171b5e57965685275f1915e241a94ff8d8919b26849d9cc37be83cd1b4"} Jul 18 11:20:09.686151 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.685830 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-41.ec2.internal" podStartSLOduration=2.685802403 podStartE2EDuration="2.685802403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:09.685188672 +0000 UTC m=+4.586952829" watchObservedRunningTime="2026-07-18 11:20:09.685802403 +0000 UTC m=+4.587566547" Jul 18 11:20:09.689636 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.689589 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6tf8c" event={"ID":"0b7c0627-1350-4093-8768-8dae93d211b9","Type":"ContainerStarted","Data":"724b6a60dbca7e1a25ef8ddd407c9624efbaaff5323eafb87cd38143c2c3cf27"} Jul 18 11:20:09.705332 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.705286 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gx6rn" event={"ID":"69157e6f-b546-4043-897f-e914056249c5","Type":"ContainerStarted","Data":"1523916f267d06673067ff301c6d51fc69914ae77cb2503d0c12e4535c61c48a"} Jul 18 11:20:09.713332 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.713234 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d8qgd" event={"ID":"6a96c874-8c04-41a0-bf56-16613a1a69c0","Type":"ContainerStarted","Data":"072cbe33241748ad5015cadbd1f03e80a2cd1983ff9979ffd637dac0844b09c3"} Jul 18 11:20:09.732309 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.732264 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" event={"ID":"4a5682f8-5600-4843-b384-5c4e7189dbef","Type":"ContainerStarted","Data":"8315e2923ced721543ee7438894087f1e5370e023f820b71eed254099eef3cc6"} Jul 18 11:20:09.739846 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.739796 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"58f91f075777b795c6308f9a5d06c9b23f1aa291f5d61a6fe1346cf460c286a6"} Jul 18 11:20:09.745508 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.745469 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-smc2t" event={"ID":"5502dd71-8ce2-4279-91ed-211ab06df23f","Type":"ContainerStarted","Data":"05c0617d4c26099e0a61d3932b055e995a0c17f776520f53361ea1d7dec4b8c1"} Jul 18 11:20:09.755107 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.754902 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k74j" event={"ID":"4af2d7f8-d600-497e-b76f-6853bf602faa","Type":"ContainerStarted","Data":"ec5af7264ac3ec94331fe6aaace7961cb202d62a903463fbb7f153f760ac9de1"} Jul 18 11:20:09.770557 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:09.770490 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-84dgn" event={"ID":"072e83ea-7610-4224-9ce9-bb0da55af877","Type":"ContainerStarted","Data":"cefb027486ff65ad0cdf55383571041d0e167fe667015c44accdc33a16dc92ca"} Jul 18 11:20:10.789101 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:10.789023 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" event={"ID":"ac67fe649f5ad8c00d895f8f51798ec7","Type":"ContainerStarted","Data":"83fad634b12e04ef7ef850c74012a23abecce203084da3c6e913016e066ca5c3"} Jul 18 11:20:11.136223 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:11.136122 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:11.136223 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:11.136195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:11.136449 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.136396 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:20:11.136449 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.136416 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:20:11.136449 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.136431 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rk7xd for pod openshift-network-diagnostics/network-check-target-trnrr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:11.136606 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.136497 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd podName:bb148327-e17d-470a-98ac-3790b9196f32 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:15.136476249 +0000 UTC m=+10.038240385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rk7xd" (UniqueName: "kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd") pod "network-check-target-trnrr" (UID: "bb148327-e17d-470a-98ac-3790b9196f32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:11.136955 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.136927 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:11.137048 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.137009 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs podName:ab36abb8-bb6e-458f-b909-277a53c37126 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:15.136991944 +0000 UTC m=+10.038756066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs") pod "network-metrics-daemon-bx2rv" (UID: "ab36abb8-bb6e-458f-b909-277a53c37126") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:11.237376 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:11.237332 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:11.237589 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.237507 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:11.237589 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.237580 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret podName:d1109b33-d650-4822-b3ab-08d1cd14d5a4 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:15.237558675 +0000 UTC m=+10.139322810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret") pod "global-pull-secret-syncer-8jxp7" (UID: "d1109b33-d650-4822-b3ab-08d1cd14d5a4") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:11.608743 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:11.608033 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:11.608743 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.608174 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:11.608743 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:11.608593 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:11.608743 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.608690 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:11.609937 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:11.609253 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:11.609937 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:11.609388 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:13.611681 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:13.610744 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:13.611681 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:13.610888 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:13.611681 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:13.611042 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:13.611681 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:13.611135 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:13.611681 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:13.611209 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:13.611681 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:13.611283 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:15.177670 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:15.177626 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:15.178180 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:15.177690 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:15.178180 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.177909 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:20:15.178180 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.177933 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:20:15.178180 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.177946 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rk7xd for pod openshift-network-diagnostics/network-check-target-trnrr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:15.178180 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.178005 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd podName:bb148327-e17d-470a-98ac-3790b9196f32 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:23.177986853 +0000 UTC m=+18.079750971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rk7xd" (UniqueName: "kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd") pod "network-check-target-trnrr" (UID: "bb148327-e17d-470a-98ac-3790b9196f32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:15.178441 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.178389 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:15.178441 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.178437 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs podName:ab36abb8-bb6e-458f-b909-277a53c37126 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:23.17842214 +0000 UTC m=+18.080186259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs") pod "network-metrics-daemon-bx2rv" (UID: "ab36abb8-bb6e-458f-b909-277a53c37126") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:15.278841 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:15.278795 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:15.279009 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.278979 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:15.279502 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.279051 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret podName:d1109b33-d650-4822-b3ab-08d1cd14d5a4 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:23.27903245 +0000 UTC m=+18.180796569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret") pod "global-pull-secret-syncer-8jxp7" (UID: "d1109b33-d650-4822-b3ab-08d1cd14d5a4") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:15.608981 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:15.608944 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:15.609197 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.609065 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:15.609448 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:15.609424 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:15.609574 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.609525 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:15.609636 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:15.609581 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:15.609690 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:15.609665 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:17.608515 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:17.608448 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:17.609017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:17.608455 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:17.609017 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:17.608594 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:17.609017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:17.608625 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:17.609017 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:17.608785 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:17.609017 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:17.608877 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:19.608710 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:19.608672 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:19.609297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:19.608686 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:19.609297 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:19.608787 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:19.609297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:19.608686 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:19.609297 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:19.608932 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:19.609297 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:19.608967 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:21.608789 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:21.608748 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:21.609283 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:21.608750 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:21.609283 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:21.608748 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:21.609283 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:21.608904 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:21.609283 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:21.608992 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:21.609283 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:21.609065 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:23.238454 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:23.238414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:23.238454 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:23.238471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:23.239101 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.238608 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:20:23.239101 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.238630 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:20:23.239101 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.238644 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rk7xd for pod openshift-network-diagnostics/network-check-target-trnrr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:23.239101 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.238671 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:23.239101 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.238710 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd podName:bb148327-e17d-470a-98ac-3790b9196f32 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:39.238690059 +0000 UTC m=+34.140454199 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rk7xd" (UniqueName: "kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd") pod "network-check-target-trnrr" (UID: "bb148327-e17d-470a-98ac-3790b9196f32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:23.239101 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.238738 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs podName:ab36abb8-bb6e-458f-b909-277a53c37126 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:39.238718999 +0000 UTC m=+34.140483117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs") pod "network-metrics-daemon-bx2rv" (UID: "ab36abb8-bb6e-458f-b909-277a53c37126") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:23.339507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:23.339470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:23.339689 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.339594 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:23.339689 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.339667 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret podName:d1109b33-d650-4822-b3ab-08d1cd14d5a4 nodeName:}" failed. No retries permitted until 2026-07-18 11:20:39.339647506 +0000 UTC m=+34.241411644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret") pod "global-pull-secret-syncer-8jxp7" (UID: "d1109b33-d650-4822-b3ab-08d1cd14d5a4") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:23.608174 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:23.608144 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:23.608370 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:23.608175 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:23.608370 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:23.608196 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:23.608370 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.608325 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:23.608542 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.608389 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:23.608542 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:23.608439 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:25.609023 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:25.608985 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:25.609563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:25.609110 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:25.609563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:25.609127 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:25.609563 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:25.609162 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:25.609563 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:25.609203 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:25.609563 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:25.609294 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:27.608752 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:27.608718 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:27.609249 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:27.608718 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:27.609249 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:27.608848 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:27.609249 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:27.608855 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:27.609249 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:27.608941 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:27.609249 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:27.609044 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:28.859140 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.858718 2585 generic.go:356] "Generic (PLEG): container finished" podID="38ac6113-5bbc-4894-a994-2436d51392e3" containerID="7173e3236bb2cc2a476b9ae4edb3be6ebcbd9724d43ea13306045f2331588e13" exitCode=0 Jul 18 11:20:28.859984 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.858769 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerDied","Data":"7173e3236bb2cc2a476b9ae4edb3be6ebcbd9724d43ea13306045f2331588e13"} Jul 18 11:20:28.861595 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.861563 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6tf8c" event={"ID":"0b7c0627-1350-4093-8768-8dae93d211b9","Type":"ContainerStarted","Data":"f7e7153cc91b7b4ab132c4531fc7ddd9286122b48f2bc60e4ee8aef6ed67d197"} Jul 18 11:20:28.863300 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.863272 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gx6rn" event={"ID":"69157e6f-b546-4043-897f-e914056249c5","Type":"ContainerStarted","Data":"52b347ba338d6682b739633db0a9dddada49bb59d4156287f50266970029969f"} Jul 18 11:20:28.865143 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.865118 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d8qgd" event={"ID":"6a96c874-8c04-41a0-bf56-16613a1a69c0","Type":"ContainerStarted","Data":"8ba39d0d521ab83f12fb6c4bca47071d9e534b37cf662b5f84b79e40810d0bae"} Jul 18 11:20:28.866967 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.866937 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" event={"ID":"4a5682f8-5600-4843-b384-5c4e7189dbef","Type":"ContainerStarted","Data":"9764863fbf136b6dd1e8ea4c13e3f34ce1e5519fc1cc72107ef37369f5ca7f0d"} Jul 18 11:20:28.870621 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.870598 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"ed70f6501cfcf63c424169af11814ace87a5609b8d0fb29e9ac8cd44c749feca"} Jul 18 11:20:28.870718 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.870677 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"74a98ee83986b4ae8aaade2ba886c9a4d9c1e86c327fc3e1c9063c30b5d74bd6"} Jul 18 11:20:28.870771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.870728 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"88749b0474b855dc0e498ff82ddda9bad2d4f5e416f5d407ff3c9060a09f750f"} Jul 18 11:20:28.870820 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.870775 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"c892f4a81b0aa59d0db24b9a56b267e524a5f8b82a2adb03cb14d5679de334a0"} Jul 18 11:20:28.870872 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.870823 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"13bf32a7ff5d84be7e66eacff950e0ef766e02f550b1f75ef8cd64e3af0dff95"} Jul 18 11:20:28.870948 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.870871 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"110addb54eb1238961a455115fd010a591f815c899e4c769bae01fc1f2d9f618"} Jul 18 11:20:28.872281 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.872259 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k74j" event={"ID":"4af2d7f8-d600-497e-b76f-6853bf602faa","Type":"ContainerStarted","Data":"afea8324a5afb3e037bf6292472da6b4e8d699abde5808c13b6fa62d545b5bac"} Jul 18 11:20:28.873939 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.873918 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-84dgn" event={"ID":"072e83ea-7610-4224-9ce9-bb0da55af877","Type":"ContainerStarted","Data":"78ea4b0a020a0608e763c9754048a8352e0335cc0946fdb91c2115d9703d793c"} Jul 18 11:20:28.881052 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.881010 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-41.ec2.internal" podStartSLOduration=21.88099615 podStartE2EDuration="21.88099615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:10.80777458 +0000 UTC m=+5.709538747" watchObservedRunningTime="2026-07-18 11:20:28.88099615 +0000 UTC m=+23.782760293" Jul 18 11:20:28.894412 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.894356 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gx6rn" podStartSLOduration=3.638967591 podStartE2EDuration="22.894336523s" totalImagesPullingTime="19.255368932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:28.893673593 +0000 UTC m=+23.795437732" watchObservedRunningTime="2026-07-18 11:20:28.894336523 +0000 UTC m=+23.796100662" Jul 18 11:20:28.910563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.910512 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6tf8c" podStartSLOduration=3.634821186 podStartE2EDuration="22.910494928s" totalImagesPullingTime="19.275673742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:28.909669991 +0000 UTC m=+23.811434128" watchObservedRunningTime="2026-07-18 11:20:28.910494928 +0000 UTC m=+23.812259065" Jul 18 11:20:28.927514 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.927459 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-84dgn" podStartSLOduration=3.602385641 podStartE2EDuration="22.927438353s" totalImagesPullingTime="19.325052712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:28.927219125 +0000 UTC m=+23.828983264" watchObservedRunningTime="2026-07-18 11:20:28.927438353 +0000 UTC m=+23.829202492" Jul 18 11:20:28.945462 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.945395 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4k74j" podStartSLOduration=3.581793208 podStartE2EDuration="22.945374592s" totalImagesPullingTime="19.363581384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:28.944569026 +0000 UTC m=+23.846333179" watchObservedRunningTime="2026-07-18 11:20:28.945374592 +0000 UTC m=+23.847138732" Jul 18 11:20:28.961121 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:28.961074 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d8qgd" podStartSLOduration=3.676003985 podStartE2EDuration="22.961057074s" totalImagesPullingTime="19.285053089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:28.960908064 +0000 UTC m=+23.862672201" watchObservedRunningTime="2026-07-18 11:20:28.961057074 +0000 UTC m=+23.862821212" Jul 18 11:20:29.090196 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.090140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:29.174084 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.173981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:29.476577 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.476548 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d8qgd_6a96c874-8c04-41a0-bf56-16613a1a69c0/dns-node-resolver/0.log" Jul 18 11:20:29.502015 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.501983 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:29.529004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.528971 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 11:20:29.533591 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.533500 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T11:20:29.528992779Z","UUID":"f6644cba-fb8c-455b-baba-c8e051688fb0","Handler":null,"Name":"","Endpoint":""} Jul 18 11:20:29.536986 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.535872 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 11:20:29.536986 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.535955 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 11:20:29.602601 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.602562 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:29.608484 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.608454 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:29.608685 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.608454 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:29.608685 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:29.608596 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:29.608685 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.608454 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:29.608844 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:29.608692 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:29.608844 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:29.608806 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:29.787336 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.787306 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:29.878104 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.878066 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" event={"ID":"4a5682f8-5600-4843-b384-5c4e7189dbef","Type":"ContainerStarted","Data":"ba51ab692a52b21bfb59e43cf74b02bac1d5db6b21546956006894dff1e7b03a"} Jul 18 11:20:29.879782 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.879701 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-smc2t" event={"ID":"5502dd71-8ce2-4279-91ed-211ab06df23f","Type":"ContainerStarted","Data":"adbdf1ade18918ac4b5cf1cf3240773246628fcf2f7119a00b9d75b4fad37480"} Jul 18 11:20:29.888481 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.888450 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:29.897811 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:29.897760 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-smc2t" podStartSLOduration=4.613281538 podStartE2EDuration="23.897744949s" totalImagesPullingTime="19.284463411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:29.89701629 +0000 UTC m=+24.798780430" watchObservedRunningTime="2026-07-18 11:20:29.897744949 +0000 UTC m=+24.799509114" Jul 18 11:20:30.085704 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:30.085599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:30.170803 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:30.170768 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:30.275401 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:30.275371 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6tf8c_0b7c0627-1350-4093-8768-8dae93d211b9/node-ca/0.log" Jul 18 11:20:30.884515 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:30.884437 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" event={"ID":"4a5682f8-5600-4843-b384-5c4e7189dbef","Type":"ContainerStarted","Data":"dfb7c513afedbd4e7269b04896c56ad8370be15e5d13d751887f9ee2ef5e995d"} Jul 18 11:20:30.903602 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:30.903555 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpr8n" podStartSLOduration=3.3549658190000002 podStartE2EDuration="24.903537025s" totalImagesPullingTime="21.548571206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:30.902916531 +0000 UTC m=+25.804680672" watchObservedRunningTime="2026-07-18 11:20:30.903537025 +0000 UTC m=+25.805301162" Jul 18 11:20:31.293150 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.293114 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:31.293799 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.293777 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:31.395795 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.395748 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:31.444354 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.444317 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:31.607941 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.607857 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:31.607941 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.607899 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:31.607941 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.607857 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:31.608273 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:31.608013 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:31.608273 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:31.608102 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:31.608273 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:31.608184 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:31.890658 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.890561 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"e1a198ae84d9994b2f2d9e887851fdca30c15c194e58bb0fc7b9a50e360cffda"} Jul 18 11:20:31.891113 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.891035 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:31.891597 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:31.891574 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gx6rn" Jul 18 11:20:32.205094 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:32.204970 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:32.730182 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:32.729990 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:33.608966 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.608722 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:33.609838 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.608727 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:33.609838 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:33.609024 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:33.609838 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.608786 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:33.609838 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:33.609092 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:33.609838 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:33.609149 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:33.899344 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.899261 2585 generic.go:356] "Generic (PLEG): container finished" podID="38ac6113-5bbc-4894-a994-2436d51392e3" containerID="ec60ea406391019a56928ba1a5d9f4c23b0dcbfad63906580820ee248c6fa43a" exitCode=0 Jul 18 11:20:33.899522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.899351 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerDied","Data":"ec60ea406391019a56928ba1a5d9f4c23b0dcbfad63906580820ee248c6fa43a"} Jul 18 11:20:33.903471 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.903445 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" event={"ID":"22dac10b-2945-4cd3-9acc-5a5a2a40c808","Type":"ContainerStarted","Data":"05dff71cbbc958189175d0c4c3c0c042ce23857684a3acaf4929f0b14ed37bb0"} Jul 18 11:20:33.903781 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.903766 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:33.919713 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.919686 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:33.950968 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:33.950916 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" podStartSLOduration=8.554187306 podStartE2EDuration="27.950899115s" totalImagesPullingTime="19.396711809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:33.948876998 +0000 UTC m=+28.850641135" watchObservedRunningTime="2026-07-18 11:20:33.950899115 +0000 UTC m=+28.852663244" Jul 18 11:20:34.866587 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:34.866550 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:34.908148 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:34.908116 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerStarted","Data":"104c1618e66c835352035a236532341317e05062fc183df237b8af5c77c0ee11"} Jul 18 11:20:34.908599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:34.908569 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:34.908707 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:34.908642 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:34.923358 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:34.923332 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:20:35.608553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:35.608513 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:35.608744 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:35.608599 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:35.608744 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:35.608620 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:35.608744 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:35.608714 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:35.608858 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:35.608776 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:35.608910 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:35.608863 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:35.912534 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:35.912447 2585 generic.go:356] "Generic (PLEG): container finished" podID="38ac6113-5bbc-4894-a994-2436d51392e3" containerID="104c1618e66c835352035a236532341317e05062fc183df237b8af5c77c0ee11" exitCode=0 Jul 18 11:20:35.913023 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:35.912524 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerDied","Data":"104c1618e66c835352035a236532341317e05062fc183df237b8af5c77c0ee11"} Jul 18 11:20:36.869845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:36.869814 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:36.971202 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:36.971175 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:37.140814 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.140724 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:37.241818 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.241782 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:37.343271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.343229 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:37.401419 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.401332 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:37.566865 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.566832 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:37.608796 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.608773 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:37.608913 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.608802 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:37.608913 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.608773 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:37.608913 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:37.608879 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:37.609029 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:37.608991 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:37.609099 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:37.609082 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:37.919270 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.919236 2585 generic.go:356] "Generic (PLEG): container finished" podID="38ac6113-5bbc-4894-a994-2436d51392e3" containerID="b68df9eec42fe84a7a66ad828facba7a147734747e427e8e5daf0763e88dae7a" exitCode=0 Jul 18 11:20:37.919455 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:37.919285 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerDied","Data":"b68df9eec42fe84a7a66ad828facba7a147734747e427e8e5daf0763e88dae7a"} Jul 18 11:20:38.131899 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:38.131861 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:38.231997 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:38.231912 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:38.284228 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:38.284199 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:38.570200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:38.570166 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:38.972090 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:38.972009 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:39.172513 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.172466 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:39.256269 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.256219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:39.256446 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.256309 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:39.256446 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.256416 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:39.256581 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.256478 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs podName:ab36abb8-bb6e-458f-b909-277a53c37126 nodeName:}" failed. No retries permitted until 2026-07-18 11:21:11.256458002 +0000 UTC m=+66.158222119 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs") pod "network-metrics-daemon-bx2rv" (UID: "ab36abb8-bb6e-458f-b909-277a53c37126") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:20:39.256581 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.256414 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:20:39.256581 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.256527 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:20:39.256581 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.256538 2585 projected.go:196] Error preparing data for projected volume kube-api-access-rk7xd for pod openshift-network-diagnostics/network-check-target-trnrr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:39.256803 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.256585 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd podName:bb148327-e17d-470a-98ac-3790b9196f32 nodeName:}" failed. No retries permitted until 2026-07-18 11:21:11.256571389 +0000 UTC m=+66.158335505 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rk7xd" (UniqueName: "kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd") pod "network-check-target-trnrr" (UID: "bb148327-e17d-470a-98ac-3790b9196f32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:20:39.357573 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.357528 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:39.357754 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.357675 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:39.357754 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.357750 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret podName:d1109b33-d650-4822-b3ab-08d1cd14d5a4 nodeName:}" failed. No retries permitted until 2026-07-18 11:21:11.357730414 +0000 UTC m=+66.259494537 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret") pod "global-pull-secret-syncer-8jxp7" (UID: "d1109b33-d650-4822-b3ab-08d1cd14d5a4") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:20:39.374067 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.374010 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:39.562168 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.562086 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:39.608173 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.608138 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:39.608367 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.608257 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:39.608367 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.608285 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:39.608367 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:39.608345 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:39.608528 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.608425 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:39.608589 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:39.608530 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:40.367526 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:40.367493 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:40.770576 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:40.770537 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:40.871243 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:40.871203 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:41.166016 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.165931 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:41.218818 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.218768 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:41.572912 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.572856 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:41.608135 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.608102 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:41.608303 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.608102 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:41.608303 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:41.608224 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:41.608424 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:41.608337 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:41.608424 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.608103 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:41.608517 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:41.608420 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:41.674494 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.674452 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:41.724430 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.724394 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:41.779838 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:41.779800 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:42.163200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:42.163158 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:42.762061 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:42.762030 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:42.933234 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:42.933200 2585 generic.go:356] "Generic (PLEG): container finished" podID="38ac6113-5bbc-4894-a994-2436d51392e3" containerID="825c930164ae45ffa3e1ded503033b6af8462ce941bee3003acd120d8fcc7906" exitCode=0 Jul 18 11:20:42.933431 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:42.933260 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerDied","Data":"825c930164ae45ffa3e1ded503033b6af8462ce941bee3003acd120d8fcc7906"} Jul 18 11:20:42.963653 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:42.963606 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:43.162710 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:43.162667 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:43.384497 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:43.384412 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:43.608640 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:43.608607 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:43.608821 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:43.608614 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:43.608821 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:43.608720 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:43.608821 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:43.608776 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:43.608821 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:43.608620 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:43.609060 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:43.608930 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:43.938021 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:43.937991 2585 generic.go:356] "Generic (PLEG): container finished" podID="38ac6113-5bbc-4894-a994-2436d51392e3" containerID="48e2f6d52256ba11ff6b8b28a1ea3a13f5a47701d2c9d882fd91819306afdaf8" exitCode=0 Jul 18 11:20:43.938438 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:43.938056 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerDied","Data":"48e2f6d52256ba11ff6b8b28a1ea3a13f5a47701d2c9d882fd91819306afdaf8"} Jul 18 11:20:44.365950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:44.365913 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:44.562811 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:44.562769 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:44.764870 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:44.764841 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:44.943292 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:44.943261 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" event={"ID":"38ac6113-5bbc-4894-a994-2436d51392e3","Type":"ContainerStarted","Data":"51b6f048c587a6698a8f03e486e4483ccd24e067b38ea59857fb496b8581214b"} Jul 18 11:20:44.966848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:44.966799 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2bbjc" podStartSLOduration=9.408591178 podStartE2EDuration="38.966783413s" totalImagesPullingTime="29.558192235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:20:44.965137748 +0000 UTC m=+39.866901886" watchObservedRunningTime="2026-07-18 11:20:44.966783413 +0000 UTC m=+39.868547552" Jul 18 11:20:44.969625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:44.969590 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:45.165255 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:45.165170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:45.609501 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:45.609464 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:45.609702 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:45.609570 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:45.609702 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:45.609576 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:45.609702 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:45.609647 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:45.609702 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:45.609656 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:45.609927 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:45.609734 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:46.166837 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:46.166804 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:46.564512 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:46.564470 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:46.767912 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:46.767850 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:46.962912 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:46.962816 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:47.367177 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:47.367145 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:47.569038 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:47.569001 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:47.608062 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:47.608033 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:47.608165 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:47.608072 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:47.608165 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:47.608036 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:47.608165 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:47.608137 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:47.608262 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:47.608238 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:47.608358 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:47.608339 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:49.608421 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:49.608379 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:49.608421 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:49.608426 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:49.608855 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:49.608401 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:49.608855 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:49.608499 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:49.608855 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:49.608571 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:49.608855 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:49.608654 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:51.607847 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:51.607807 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:51.608367 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:51.607807 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:51.608367 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:51.607940 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:51.608367 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:51.607820 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:51.608367 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:51.608005 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:51.608367 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:51.608090 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:52.687556 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:52.687515 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:52.774042 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:52.773999 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:52.875292 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:52.875254 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:52.974141 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:52.974058 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:53.072581 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:53.072544 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:53.608819 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:53.608779 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:53.609033 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:53.608793 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:53.609033 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:53.608895 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:53.609147 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:53.609084 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:53.609215 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:53.609190 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:53.609307 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:53.609288 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:53.973414 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:53.973375 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:54.074123 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:54.074080 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:54.182782 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:54.182748 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8jxp7"] Jul 18 11:20:54.182983 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:54.182878 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:54.183059 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:54.183012 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:54.186386 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:54.186332 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bx2rv"] Jul 18 11:20:54.186528 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:54.186477 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:54.186623 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:54.186594 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:54.187214 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:54.187188 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-trnrr"] Jul 18 11:20:54.187356 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:54.187276 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:54.187405 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:54.187361 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:55.608625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:55.608544 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:55.608625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:55.608616 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:55.609120 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:55.608627 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:55.609120 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:55.608699 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:55.609120 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:55.608743 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:55.609120 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:55.608799 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:56.612951 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:56.612913 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:56.661745 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:56.661716 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:57.608634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:57.608595 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:57.608842 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:57.608603 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:57.608842 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:57.608711 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx2rv" podUID="ab36abb8-bb6e-458f-b909-277a53c37126" Jul 18 11:20:57.608842 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:57.608603 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:57.608842 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:57.608773 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-trnrr" podUID="bb148327-e17d-470a-98ac-3790b9196f32" Jul 18 11:20:57.609094 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:20:57.608845 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jxp7" podUID="d1109b33-d650-4822-b3ab-08d1cd14d5a4" Jul 18 11:20:58.889594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.889517 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-41.ec2.internal" event="NodeReady" Jul 18 11:20:58.890065 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.889640 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 11:20:58.935653 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.935620 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w6rlm"] Jul 18 11:20:58.958200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.958161 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-597tz"] Jul 18 11:20:58.958383 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.958206 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:58.961077 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.960851 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 11:20:58.961077 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.960851 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 11:20:58.961824 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.961199 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-72crb\"" Jul 18 11:20:58.974158 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.974126 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w6rlm"] Jul 18 11:20:58.974303 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.974177 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-597tz"] Jul 18 11:20:58.974303 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.974193 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-89wsg"] Jul 18 11:20:58.974303 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.974220 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:58.978271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.977834 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 11:20:58.978271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.977849 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dgc9c\"" Jul 18 11:20:58.978271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.978120 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 11:20:58.978271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.978171 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 11:20:58.992258 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.992225 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-89wsg"] Jul 18 11:20:58.992409 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.992348 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:58.994634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.994604 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-config-volume\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:58.994748 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.994675 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-tmp-dir\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:58.994748 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.994701 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421f75d5-537e-4d1f-9129-7713b9851728-cert\") pod \"ingress-canary-597tz\" (UID: \"421f75d5-537e-4d1f-9129-7713b9851728\") " pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:58.994748 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.994726 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-metrics-tls\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:58.994918 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.994764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh8qr\" (UniqueName: \"kubernetes.io/projected/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-kube-api-access-xh8qr\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:58.994918 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.994810 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf72w\" (UniqueName: \"kubernetes.io/projected/421f75d5-537e-4d1f-9129-7713b9851728-kube-api-access-wf72w\") pod \"ingress-canary-597tz\" (UID: \"421f75d5-537e-4d1f-9129-7713b9851728\") " pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:58.995068 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.995033 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 11:20:58.995160 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.995140 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 11:20:58.995226 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.995147 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nhs4n\"" Jul 18 11:20:58.995226 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.995178 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 11:20:58.995226 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:58.995213 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 11:20:59.013276 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.013244 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.076344 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.076310 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.095334 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf72w\" (UniqueName: \"kubernetes.io/projected/421f75d5-537e-4d1f-9129-7713b9851728-kube-api-access-wf72w\") pod \"ingress-canary-597tz\" (UID: \"421f75d5-537e-4d1f-9129-7713b9851728\") " pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:59.095470 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095342 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08c58c0e-cea2-4795-a53a-958964e776f4-crio-socket\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.095470 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08c58c0e-cea2-4795-a53a-958964e776f4-data-volume\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.095470 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08c58c0e-cea2-4795-a53a-958964e776f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.095470 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095393 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08c58c0e-cea2-4795-a53a-958964e776f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.095470 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095415 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r7jn\" (UniqueName: \"kubernetes.io/projected/08c58c0e-cea2-4795-a53a-958964e776f4-kube-api-access-4r7jn\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.095650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095493 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-config-volume\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.095650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095563 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-tmp-dir\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.095650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095587 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421f75d5-537e-4d1f-9129-7713b9851728-cert\") pod \"ingress-canary-597tz\" (UID: \"421f75d5-537e-4d1f-9129-7713b9851728\") " pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:59.095650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095611 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-metrics-tls\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.095650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095649 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh8qr\" (UniqueName: \"kubernetes.io/projected/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-kube-api-access-xh8qr\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.095999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.095981 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-tmp-dir\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.096150 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.096133 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-config-volume\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.099820 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.099801 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421f75d5-537e-4d1f-9129-7713b9851728-cert\") pod \"ingress-canary-597tz\" (UID: \"421f75d5-537e-4d1f-9129-7713b9851728\") " pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:59.099899 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.099812 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-metrics-tls\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.103083 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.103061 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf72w\" (UniqueName: \"kubernetes.io/projected/421f75d5-537e-4d1f-9129-7713b9851728-kube-api-access-wf72w\") pod \"ingress-canary-597tz\" (UID: \"421f75d5-537e-4d1f-9129-7713b9851728\") " pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:59.103181 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.103087 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh8qr\" (UniqueName: \"kubernetes.io/projected/d796e035-c9ad-4fce-a3cf-dfcc23d3992f-kube-api-access-xh8qr\") pod \"dns-default-w6rlm\" (UID: \"d796e035-c9ad-4fce-a3cf-dfcc23d3992f\") " pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.176114 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.176023 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.196267 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196222 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08c58c0e-cea2-4795-a53a-958964e776f4-crio-socket\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.196267 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196265 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08c58c0e-cea2-4795-a53a-958964e776f4-data-volume\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.196522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196289 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08c58c0e-cea2-4795-a53a-958964e776f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.196522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08c58c0e-cea2-4795-a53a-958964e776f4-crio-socket\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.196522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08c58c0e-cea2-4795-a53a-958964e776f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.196522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196460 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r7jn\" (UniqueName: \"kubernetes.io/projected/08c58c0e-cea2-4795-a53a-958964e776f4-kube-api-access-4r7jn\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.196733 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196713 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08c58c0e-cea2-4795-a53a-958964e776f4-data-volume\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.196954 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.196934 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08c58c0e-cea2-4795-a53a-958964e776f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.199067 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.199051 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08c58c0e-cea2-4795-a53a-958964e776f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.205118 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.205091 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r7jn\" (UniqueName: \"kubernetes.io/projected/08c58c0e-cea2-4795-a53a-958964e776f4-kube-api-access-4r7jn\") pod \"insights-runtime-extractor-89wsg\" (UID: \"08c58c0e-cea2-4795-a53a-958964e776f4\") " pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.268912 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.268863 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w6rlm" Jul 18 11:20:59.278296 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.278261 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.284157 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.284124 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-597tz" Jul 18 11:20:59.301027 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.301000 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-89wsg" Jul 18 11:20:59.336082 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.335988 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.395988 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.394988 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.485116 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.485074 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w6rlm"] Jul 18 11:20:59.489772 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.489687 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-89wsg"] Jul 18 11:20:59.490700 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:59.490665 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd796e035_c9ad_4fce_a3cf_dfcc23d3992f.slice/crio-e7d0b8d22122cb002d206d0d612c16e2c1aa3f3ed7131a37193636bba1f29b4a WatchSource:0}: Error finding container e7d0b8d22122cb002d206d0d612c16e2c1aa3f3ed7131a37193636bba1f29b4a: Status 404 returned error can't find the container with id e7d0b8d22122cb002d206d0d612c16e2c1aa3f3ed7131a37193636bba1f29b4a Jul 18 11:20:59.493703 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:59.493673 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c58c0e_cea2_4795_a53a_958964e776f4.slice/crio-e4727e13d9a5f8cc9bce34151fcd73015bfcedbbfae57b5de4560f1ee17db0d1 WatchSource:0}: Error finding container e4727e13d9a5f8cc9bce34151fcd73015bfcedbbfae57b5de4560f1ee17db0d1: Status 404 returned error can't find the container with id e4727e13d9a5f8cc9bce34151fcd73015bfcedbbfae57b5de4560f1ee17db0d1 Jul 18 11:20:59.494703 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.494204 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-597tz"] Jul 18 11:20:59.498266 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:20:59.498227 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421f75d5_537e_4d1f_9129_7713b9851728.slice/crio-637de5f92c8064883bc8babb1fcd4209d9f715aa300e1650dd017bce863e6016 WatchSource:0}: Error finding container 637de5f92c8064883bc8babb1fcd4209d9f715aa300e1650dd017bce863e6016: Status 404 returned error can't find the container with id 637de5f92c8064883bc8babb1fcd4209d9f715aa300e1650dd017bce863e6016 Jul 18 11:20:59.608364 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.608323 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:20:59.608522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.608323 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:20:59.608522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.608323 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:20:59.612858 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.612829 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 11:20:59.613026 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.612910 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 11:20:59.613281 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.613265 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b49zp\"" Jul 18 11:20:59.614002 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.613985 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 11:20:59.614049 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.614025 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s78sq\"" Jul 18 11:20:59.614049 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.614040 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 11:20:59.625625 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.625596 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.723230 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.723113 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:20:59.983817 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.983729 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89wsg" event={"ID":"08c58c0e-cea2-4795-a53a-958964e776f4","Type":"ContainerStarted","Data":"e4727e13d9a5f8cc9bce34151fcd73015bfcedbbfae57b5de4560f1ee17db0d1"} Jul 18 11:20:59.985602 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.985569 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-597tz" event={"ID":"421f75d5-537e-4d1f-9129-7713b9851728","Type":"ContainerStarted","Data":"637de5f92c8064883bc8babb1fcd4209d9f715aa300e1650dd017bce863e6016"} Jul 18 11:20:59.987139 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:20:59.987104 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6rlm" event={"ID":"d796e035-c9ad-4fce-a3cf-dfcc23d3992f","Type":"ContainerStarted","Data":"e7d0b8d22122cb002d206d0d612c16e2c1aa3f3ed7131a37193636bba1f29b4a"} Jul 18 11:21:00.263548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:00.263474 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:00.657127 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:00.657034 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:00.708912 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:00.708729 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:01.764454 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:01.764350 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:01.819003 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:01.818960 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:03.494460 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.494426 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-mrw69"] Jul 18 11:21:03.510930 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.510868 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:03.606296 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.606257 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-mrw69"] Jul 18 11:21:03.606465 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.606385 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-mrw69" Jul 18 11:21:03.609823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.609539 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 18 11:21:03.609823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.609617 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 18 11:21:03.609823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.609647 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hx8lk\"" Jul 18 11:21:03.609823 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.609686 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:03.633206 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.633171 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l77lk\" (UniqueName: \"kubernetes.io/projected/0e141688-e460-49f5-87a9-44361c652b0b-kube-api-access-l77lk\") pod \"downloads-5bc67dcd46-mrw69\" (UID: \"0e141688-e460-49f5-87a9-44361c652b0b\") " pod="openshift-console/downloads-5bc67dcd46-mrw69" Jul 18 11:21:03.678953 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.678852 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:03.734018 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.733986 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l77lk\" (UniqueName: \"kubernetes.io/projected/0e141688-e460-49f5-87a9-44361c652b0b-kube-api-access-l77lk\") pod \"downloads-5bc67dcd46-mrw69\" (UID: \"0e141688-e460-49f5-87a9-44361c652b0b\") " pod="openshift-console/downloads-5bc67dcd46-mrw69" Jul 18 11:21:03.745915 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.745818 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l77lk\" (UniqueName: \"kubernetes.io/projected/0e141688-e460-49f5-87a9-44361c652b0b-kube-api-access-l77lk\") pod \"downloads-5bc67dcd46-mrw69\" (UID: \"0e141688-e460-49f5-87a9-44361c652b0b\") " pod="openshift-console/downloads-5bc67dcd46-mrw69" Jul 18 11:21:03.801297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.801255 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:03.918689 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:03.918652 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-mrw69" Jul 18 11:21:04.185561 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.185522 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:04.243745 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.243709 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5"] Jul 18 11:21:04.263405 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.263310 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:04.313783 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.313726 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5"] Jul 18 11:21:04.313997 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.313874 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.318621 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.318590 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 11:21:04.318779 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.318623 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ks2cw\"" Jul 18 11:21:04.318779 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.318677 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 11:21:04.318779 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.318628 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 18 11:21:04.318969 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.318847 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 18 11:21:04.318969 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.318957 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 11:21:04.339483 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.339433 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c69bd109-a8f7-476d-ac91-375617cc0d52-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.339483 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.339483 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69bd109-a8f7-476d-ac91-375617cc0d52-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.339712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.339540 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2f7k\" (UniqueName: \"kubernetes.io/projected/c69bd109-a8f7-476d-ac91-375617cc0d52-kube-api-access-n2f7k\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.339712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.339579 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c69bd109-a8f7-476d-ac91-375617cc0d52-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.440472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.440418 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c69bd109-a8f7-476d-ac91-375617cc0d52-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.440472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.440475 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69bd109-a8f7-476d-ac91-375617cc0d52-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.440725 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.440539 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2f7k\" (UniqueName: \"kubernetes.io/projected/c69bd109-a8f7-476d-ac91-375617cc0d52-kube-api-access-n2f7k\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.440725 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.440569 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c69bd109-a8f7-476d-ac91-375617cc0d52-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.441302 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.441279 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c69bd109-a8f7-476d-ac91-375617cc0d52-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.443141 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.443114 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c69bd109-a8f7-476d-ac91-375617cc0d52-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.443279 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.443231 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c69bd109-a8f7-476d-ac91-375617cc0d52-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.450786 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.450754 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2f7k\" (UniqueName: \"kubernetes.io/projected/c69bd109-a8f7-476d-ac91-375617cc0d52-kube-api-access-n2f7k\") pod \"prometheus-operator-5945b5cfd-tmvk5\" (UID: \"c69bd109-a8f7-476d-ac91-375617cc0d52\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.475652 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.475615 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:04.625485 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.625392 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" Jul 18 11:21:04.810799 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.810755 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:04.912276 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:04.912101 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:05.012979 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:05.012939 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:05.065298 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:05.065258 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:05.388286 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:05.388236 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:05.494402 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:05.494371 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-mrw69"] Jul 18 11:21:05.498846 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:05.498808 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e141688_e460_49f5_87a9_44361c652b0b.slice/crio-a1beda1d05693fc54749facc603936bf2969e2a468c8961c62c8b2a0a6d48e2d WatchSource:0}: Error finding container a1beda1d05693fc54749facc603936bf2969e2a468c8961c62c8b2a0a6d48e2d: Status 404 returned error can't find the container with id a1beda1d05693fc54749facc603936bf2969e2a468c8961c62c8b2a0a6d48e2d Jul 18 11:21:05.507512 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:05.507408 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5"] Jul 18 11:21:05.512012 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:05.511977 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69bd109_a8f7_476d_ac91_375617cc0d52.slice/crio-ccae378b295a787b76626e956318d7e68c1678fcd404d7d8c28a265df4de8ce0 WatchSource:0}: Error finding container ccae378b295a787b76626e956318d7e68c1678fcd404d7d8c28a265df4de8ce0: Status 404 returned error can't find the container with id ccae378b295a787b76626e956318d7e68c1678fcd404d7d8c28a265df4de8ce0 Jul 18 11:21:05.579968 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:05.579919 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:05.680434 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:05.680393 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:06.008041 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.007953 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89wsg" event={"ID":"08c58c0e-cea2-4795-a53a-958964e776f4","Type":"ContainerStarted","Data":"b6e9a104a06bd0ded6b4ec53c4cfd79f24075ddc69e655d5839ce0703fd8adc7"} Jul 18 11:21:06.013744 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.013659 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-597tz" event={"ID":"421f75d5-537e-4d1f-9129-7713b9851728","Type":"ContainerStarted","Data":"619fc9bd4a6ef9893d74bdfbeef00eb70a47b2e2018d1af9726742c7c1399a6d"} Jul 18 11:21:06.018218 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.018138 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" event={"ID":"c69bd109-a8f7-476d-ac91-375617cc0d52","Type":"ContainerStarted","Data":"ccae378b295a787b76626e956318d7e68c1678fcd404d7d8c28a265df4de8ce0"} Jul 18 11:21:06.021501 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.021341 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6rlm" event={"ID":"d796e035-c9ad-4fce-a3cf-dfcc23d3992f","Type":"ContainerStarted","Data":"058301527b09c90179a92ed7fcbd96b33b06ec9bd40d39a4d14f588e41d437ac"} Jul 18 11:21:06.021501 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.021399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w6rlm" event={"ID":"d796e035-c9ad-4fce-a3cf-dfcc23d3992f","Type":"ContainerStarted","Data":"a130d4461a030498b7a6da2bab1654c21737bb92e0e5e6d86837b1fd4461b035"} Jul 18 11:21:06.022086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.022065 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w6rlm" Jul 18 11:21:06.023907 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.023861 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-mrw69" event={"ID":"0e141688-e460-49f5-87a9-44361c652b0b","Type":"ContainerStarted","Data":"a1beda1d05693fc54749facc603936bf2969e2a468c8961c62c8b2a0a6d48e2d"} Jul 18 11:21:06.032286 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.032188 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-597tz" podStartSLOduration=2.209696662 podStartE2EDuration="8.032168303s" totalImagesPullingTime="5.822471641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:06.031109728 +0000 UTC m=+60.932873869" watchObservedRunningTime="2026-07-18 11:21:06.032168303 +0000 UTC m=+60.933932541" Jul 18 11:21:06.931236 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.931186 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6q55b" Jul 18 11:21:06.960222 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:06.960150 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w6rlm" podStartSLOduration=3.134488734 podStartE2EDuration="8.960124863s" totalImagesPullingTime="5.825636129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:06.05034262 +0000 UTC m=+60.952106760" watchObservedRunningTime="2026-07-18 11:21:06.960124863 +0000 UTC m=+61.861888980" Jul 18 11:21:07.037613 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:07.037561 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:07.089991 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:07.089910 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:07.151243 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:07.151171 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:07.546126 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:07.546087 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:08.036044 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:08.036000 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" event={"ID":"c69bd109-a8f7-476d-ac91-375617cc0d52","Type":"ContainerStarted","Data":"29cb3cc08e8682290896d0d1df882b3ee48bb816dfea697a6f3f78c29695b8b5"} Jul 18 11:21:08.036518 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:08.036104 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" event={"ID":"c69bd109-a8f7-476d-ac91-375617cc0d52","Type":"ContainerStarted","Data":"b9af43e46c27e6096822d8d99382018ac941b6298791205853322fe3e4d94b8b"} Jul 18 11:21:08.055796 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:08.055721 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-tmvk5" podStartSLOduration=2.399894927 podStartE2EDuration="4.055701236s" totalImagesPullingTime="1.655806309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:08.055372717 +0000 UTC m=+62.957136856" watchObservedRunningTime="2026-07-18 11:21:08.055701236 +0000 UTC m=+62.957465376" Jul 18 11:21:08.824853 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:08.824816 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:08.922550 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:08.922510 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:08.975493 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:08.975448 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:09.338845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.338757 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:09.547561 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.547527 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:09.581072 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.581035 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm"] Jul 18 11:21:09.591514 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.591417 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.594074 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.594041 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 18 11:21:09.594233 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.594040 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 18 11:21:09.594233 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.594046 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jvvz5\"" Jul 18 11:21:09.603397 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.603359 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm"] Jul 18 11:21:09.604082 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.604056 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kxrfk"] Jul 18 11:21:09.607748 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.607720 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.610354 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.610326 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 11:21:09.610354 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.610344 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kw9x4\"" Jul 18 11:21:09.610626 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.610327 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 11:21:09.610684 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.610643 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 11:21:09.611986 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.611962 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-n5l4t"] Jul 18 11:21:09.615228 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.615203 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.617603 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.617581 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 18 11:21:09.617916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.617873 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tmj47\"" Jul 18 11:21:09.618054 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.618033 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 18 11:21:09.618245 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.618225 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 18 11:21:09.620170 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.620146 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-n5l4t"] Jul 18 11:21:09.638390 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.638347 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:09.686431 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686395 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.686431 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44v8q\" (UniqueName: \"kubernetes.io/projected/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-api-access-44v8q\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.686634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686465 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.686634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686515 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmsrd\" (UniqueName: \"kubernetes.io/projected/a7154b70-e5dd-4f5a-b51a-41ad727aab16-kube-api-access-pmsrd\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686572 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-tls\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686627 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-accelerators-collector-config\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686790 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686649 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/65929cc1-658b-4d72-81b9-73f85a46a9eb-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.686790 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686686 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-sys\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686790 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686710 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.686790 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686748 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.686790 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686778 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7154b70-e5dd-4f5a-b51a-41ad727aab16-metrics-client-ca\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686805 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-textfile\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686859 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-wtmp\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686912 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-root\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686934 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.686994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686954 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f743c850-a5e0-4607-ba56-16bb19c1d0f8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.686994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.686979 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.687193 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.687002 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65929cc1-658b-4d72-81b9-73f85a46a9eb-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.687193 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.687018 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drk9l\" (UniqueName: \"kubernetes.io/projected/f743c850-a5e0-4607-ba56-16bb19c1d0f8-kube-api-access-drk9l\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.788139 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788097 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-wtmp\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.788288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788144 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-root\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.788288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788169 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.788288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788189 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f743c850-a5e0-4607-ba56-16bb19c1d0f8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.788288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.788288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788238 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65929cc1-658b-4d72-81b9-73f85a46a9eb-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.788288 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788264 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drk9l\" (UniqueName: \"kubernetes.io/projected/f743c850-a5e0-4607-ba56-16bb19c1d0f8-kube-api-access-drk9l\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.788598 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788288 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-wtmp\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788286 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-root\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788293 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44v8q\" (UniqueName: \"kubernetes.io/projected/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-api-access-44v8q\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788863 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmsrd\" (UniqueName: \"kubernetes.io/projected/a7154b70-e5dd-4f5a-b51a-41ad727aab16-kube-api-access-pmsrd\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788931 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-tls\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.788983 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-accelerators-collector-config\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789017 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/65929cc1-658b-4d72-81b9-73f85a46a9eb-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-sys\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789069 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789100 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789125 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789132 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7154b70-e5dd-4f5a-b51a-41ad727aab16-metrics-client-ca\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789167 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-textfile\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.789340 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789196 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f743c850-a5e0-4607-ba56-16bb19c1d0f8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789803 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-accelerators-collector-config\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.789834 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7154b70-e5dd-4f5a-b51a-41ad727aab16-metrics-client-ca\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:21:09.789842 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:21:09.789923 2585 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:21:09.789943 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-tls podName:a7154b70-e5dd-4f5a-b51a-41ad727aab16 nodeName:}" failed. No retries permitted until 2026-07-18 11:21:10.289924514 +0000 UTC m=+65.191688631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-tls") pod "node-exporter-kxrfk" (UID: "a7154b70-e5dd-4f5a-b51a-41ad727aab16") : secret "node-exporter-tls" not found Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:21:09.790050 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-tls podName:f743c850-a5e0-4607-ba56-16bb19c1d0f8 nodeName:}" failed. No retries permitted until 2026-07-18 11:21:10.290030542 +0000 UTC m=+65.191794671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-z2tpm" (UID: "f743c850-a5e0-4607-ba56-16bb19c1d0f8") : secret "openshift-state-metrics-tls" not found Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.790050 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-textfile\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.790177 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.790069 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65929cc1-658b-4d72-81b9-73f85a46a9eb-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.790581 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.790281 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7154b70-e5dd-4f5a-b51a-41ad727aab16-sys\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.790581 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.790496 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/65929cc1-658b-4d72-81b9-73f85a46a9eb-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.791413 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.791384 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.792026 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.791999 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.792178 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.792159 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.792990 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.792969 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.799697 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.799664 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmsrd\" (UniqueName: \"kubernetes.io/projected/a7154b70-e5dd-4f5a-b51a-41ad727aab16-kube-api-access-pmsrd\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:09.799864 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.799841 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44v8q\" (UniqueName: \"kubernetes.io/projected/65929cc1-658b-4d72-81b9-73f85a46a9eb-kube-api-access-44v8q\") pod \"kube-state-metrics-56c5594674-n5l4t\" (UID: \"65929cc1-658b-4d72-81b9-73f85a46a9eb\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:09.800154 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.800132 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drk9l\" (UniqueName: \"kubernetes.io/projected/f743c850-a5e0-4607-ba56-16bb19c1d0f8-kube-api-access-drk9l\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:09.835072 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.835004 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:09.884869 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.884748 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:09.929332 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:09.929294 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" Jul 18 11:21:10.049331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.049295 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89wsg" event={"ID":"08c58c0e-cea2-4795-a53a-958964e776f4","Type":"ContainerStarted","Data":"5cf483944d6995f3df39760412edb8d2cfa120478c14f23345cfdaeaa0dfb567"} Jul 18 11:21:10.083260 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.083227 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-n5l4t"] Jul 18 11:21:10.086428 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:10.086392 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65929cc1_658b_4d72_81b9_73f85a46a9eb.slice/crio-710c37465b4d6653eba6b54036b60a31a1e474e0c6859756d7c294b7b6c7bd12 WatchSource:0}: Error finding container 710c37465b4d6653eba6b54036b60a31a1e474e0c6859756d7c294b7b6c7bd12: Status 404 returned error can't find the container with id 710c37465b4d6653eba6b54036b60a31a1e474e0c6859756d7c294b7b6c7bd12 Jul 18 11:21:10.182246 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.182159 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:10.293765 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.293720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:10.293983 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.293918 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-tls\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:10.296543 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.296511 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7154b70-e5dd-4f5a-b51a-41ad727aab16-node-exporter-tls\") pod \"node-exporter-kxrfk\" (UID: \"a7154b70-e5dd-4f5a-b51a-41ad727aab16\") " pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:10.296687 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.296563 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f743c850-a5e0-4607-ba56-16bb19c1d0f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-z2tpm\" (UID: \"f743c850-a5e0-4607-ba56-16bb19c1d0f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:10.500647 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.500555 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" Jul 18 11:21:10.522040 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.521114 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kxrfk" Jul 18 11:21:10.541600 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:10.541564 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7154b70_e5dd_4f5a_b51a_41ad727aab16.slice/crio-5c58edccad3a9b3e66d5bc3d3668cabcea90305773eab18e188fa1db9b9a3574 WatchSource:0}: Error finding container 5c58edccad3a9b3e66d5bc3d3668cabcea90305773eab18e188fa1db9b9a3574: Status 404 returned error can't find the container with id 5c58edccad3a9b3e66d5bc3d3668cabcea90305773eab18e188fa1db9b9a3574 Jul 18 11:21:10.570162 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.570121 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:10.660530 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.660467 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:10.682628 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.682569 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm"] Jul 18 11:21:10.687404 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:10.687362 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf743c850_a5e0_4607_ba56_16bb19c1d0f8.slice/crio-1b1333f5ed385b2108438b69ae46429a8a6e8602d46d41222db9767b1b9e5dc3 WatchSource:0}: Error finding container 1b1333f5ed385b2108438b69ae46429a8a6e8602d46d41222db9767b1b9e5dc3: Status 404 returned error can't find the container with id 1b1333f5ed385b2108438b69ae46429a8a6e8602d46d41222db9767b1b9e5dc3 Jul 18 11:21:10.972930 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:10.972764 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:11.056095 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.055955 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" event={"ID":"65929cc1-658b-4d72-81b9-73f85a46a9eb","Type":"ContainerStarted","Data":"710c37465b4d6653eba6b54036b60a31a1e474e0c6859756d7c294b7b6c7bd12"} Jul 18 11:21:11.058755 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.058575 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" event={"ID":"f743c850-a5e0-4607-ba56-16bb19c1d0f8","Type":"ContainerStarted","Data":"54d2aa3bcf3d2d89a1b8d7256f39fe08996b17c79e931c6071020fe60eae86dd"} Jul 18 11:21:11.058755 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.058649 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" event={"ID":"f743c850-a5e0-4607-ba56-16bb19c1d0f8","Type":"ContainerStarted","Data":"e0d05e23d046a55188d96dc4be405832ca3e1c3f1809fe7bb5546f3528de3b56"} Jul 18 11:21:11.058755 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.058680 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" event={"ID":"f743c850-a5e0-4607-ba56-16bb19c1d0f8","Type":"ContainerStarted","Data":"1b1333f5ed385b2108438b69ae46429a8a6e8602d46d41222db9767b1b9e5dc3"} Jul 18 11:21:11.060267 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.060200 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrfk" event={"ID":"a7154b70-e5dd-4f5a-b51a-41ad727aab16","Type":"ContainerStarted","Data":"5c58edccad3a9b3e66d5bc3d3668cabcea90305773eab18e188fa1db9b9a3574"} Jul 18 11:21:11.305496 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.305453 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:21:11.305709 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.305531 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:21:11.308663 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.308574 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 11:21:11.309699 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.309674 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 11:21:11.318573 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.318543 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 11:21:11.321142 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.321077 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab36abb8-bb6e-458f-b909-277a53c37126-metrics-certs\") pod \"network-metrics-daemon-bx2rv\" (UID: \"ab36abb8-bb6e-458f-b909-277a53c37126\") " pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:21:11.327563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.327483 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b49zp\"" Jul 18 11:21:11.329899 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.329750 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk7xd\" (UniqueName: \"kubernetes.io/projected/bb148327-e17d-470a-98ac-3790b9196f32-kube-api-access-rk7xd\") pod \"network-check-target-trnrr\" (UID: \"bb148327-e17d-470a-98ac-3790b9196f32\") " pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:21:11.337614 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.337369 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx2rv" Jul 18 11:21:11.406822 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.406785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:21:11.410345 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.410280 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 11:21:11.419878 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.419849 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1109b33-d650-4822-b3ab-08d1cd14d5a4-original-pull-secret\") pod \"global-pull-secret-syncer-8jxp7\" (UID: \"d1109b33-d650-4822-b3ab-08d1cd14d5a4\") " pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:21:11.501738 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.501675 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:11.559200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.559101 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65f7c6c966-wqwxs"] Jul 18 11:21:11.563079 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.563045 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.566847 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.566636 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 18 11:21:11.567058 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.567040 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 18 11:21:11.567250 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.567233 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 18 11:21:11.567360 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.567338 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cci1e1mvijiq5\"" Jul 18 11:21:11.567457 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.567441 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kcwpj\"" Jul 18 11:21:11.567646 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.567631 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 18 11:21:11.567704 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.567682 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 18 11:21:11.575716 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.575680 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65f7c6c966-wqwxs"] Jul 18 11:21:11.579199 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.579166 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:11.621703 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.621674 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s78sq\"" Jul 18 11:21:11.629298 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.629261 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:21:11.630503 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.630473 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jxp7" Jul 18 11:21:11.635781 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.635747 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:11.709773 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.709730 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.709982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.709789 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.709982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.709824 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-grpc-tls\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.709982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.709853 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md7z6\" (UniqueName: \"kubernetes.io/projected/686b7d6c-37f5-4082-8ed4-49b0d648fa18-kube-api-access-md7z6\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.709982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.709942 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.710211 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.710003 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-tls\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.710211 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.710070 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.710211 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.710098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/686b7d6c-37f5-4082-8ed4-49b0d648fa18-metrics-client-ca\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811582 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-tls\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811582 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811571 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811600 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/686b7d6c-37f5-4082-8ed4-49b0d648fa18-metrics-client-ca\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811653 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811684 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811709 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-grpc-tls\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811737 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md7z6\" (UniqueName: \"kubernetes.io/projected/686b7d6c-37f5-4082-8ed4-49b0d648fa18-kube-api-access-md7z6\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.811809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.811766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.813308 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.813272 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/686b7d6c-37f5-4082-8ed4-49b0d648fa18-metrics-client-ca\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.817238 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.817177 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-grpc-tls\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.818141 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.818111 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.818665 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.818500 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.818665 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.818642 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-tls\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.819079 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.819036 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.819327 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.819302 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/686b7d6c-37f5-4082-8ed4-49b0d648fa18-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.823370 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.823320 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md7z6\" (UniqueName: \"kubernetes.io/projected/686b7d6c-37f5-4082-8ed4-49b0d648fa18-kube-api-access-md7z6\") pod \"thanos-querier-65f7c6c966-wqwxs\" (UID: \"686b7d6c-37f5-4082-8ed4-49b0d648fa18\") " pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:11.880472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:11.880328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:12.216577 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:12.216379 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bx2rv"] Jul 18 11:21:12.254561 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:12.253746 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-trnrr"] Jul 18 11:21:12.274965 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:12.272839 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8jxp7"] Jul 18 11:21:12.277725 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:12.277622 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65f7c6c966-wqwxs"] Jul 18 11:21:12.282278 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:12.282173 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1109b33_d650_4822_b3ab_08d1cd14d5a4.slice/crio-82bdb6935018a1b13dabd7264b4880415c6ccb3d7b0a2f6b67cb361ba5a07450 WatchSource:0}: Error finding container 82bdb6935018a1b13dabd7264b4880415c6ccb3d7b0a2f6b67cb361ba5a07450: Status 404 returned error can't find the container with id 82bdb6935018a1b13dabd7264b4880415c6ccb3d7b0a2f6b67cb361ba5a07450 Jul 18 11:21:12.530942 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:12.530872 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:12.638168 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:12.638047 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:12.735739 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:12.735576 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:13.080906 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.080847 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" event={"ID":"686b7d6c-37f5-4082-8ed4-49b0d648fa18","Type":"ContainerStarted","Data":"5adced1670c858d9e040d1a31cf3a302358b6c03ca2dd089808bbd5d1cab8285"} Jul 18 11:21:13.088401 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.088353 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" event={"ID":"65929cc1-658b-4d72-81b9-73f85a46a9eb","Type":"ContainerStarted","Data":"d08f0e952b2f52da8a70f44708d4a620a050231b6a51ce71e03ba5b88139336a"} Jul 18 11:21:13.088588 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.088439 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" event={"ID":"65929cc1-658b-4d72-81b9-73f85a46a9eb","Type":"ContainerStarted","Data":"46397ba2e6077af5beec6c92895589f0007d7347644b6df6d32584b32c0e6175"} Jul 18 11:21:13.088588 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.088469 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" event={"ID":"65929cc1-658b-4d72-81b9-73f85a46a9eb","Type":"ContainerStarted","Data":"84efdc62ef8c228a47c1eae7fe8cf9e2ddf4b55dadf37b3b017863c7267be115"} Jul 18 11:21:13.090037 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.089987 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8jxp7" event={"ID":"d1109b33-d650-4822-b3ab-08d1cd14d5a4","Type":"ContainerStarted","Data":"82bdb6935018a1b13dabd7264b4880415c6ccb3d7b0a2f6b67cb361ba5a07450"} Jul 18 11:21:13.092010 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.091795 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx2rv" event={"ID":"ab36abb8-bb6e-458f-b909-277a53c37126","Type":"ContainerStarted","Data":"eacaa0a11da75091679f0048d6f65211c341f6801ba00e7ae245a0804c112f01"} Jul 18 11:21:13.095814 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.094966 2585 generic.go:356] "Generic (PLEG): container finished" podID="a7154b70-e5dd-4f5a-b51a-41ad727aab16" containerID="8dab0a9c967a56f38548eb0e73bbbbce90f1b94285d4ef482ea5af402aa4af36" exitCode=0 Jul 18 11:21:13.095814 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.095055 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrfk" event={"ID":"a7154b70-e5dd-4f5a-b51a-41ad727aab16","Type":"ContainerDied","Data":"8dab0a9c967a56f38548eb0e73bbbbce90f1b94285d4ef482ea5af402aa4af36"} Jul 18 11:21:13.097829 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.097781 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-trnrr" event={"ID":"bb148327-e17d-470a-98ac-3790b9196f32","Type":"ContainerStarted","Data":"ed9ac95b2be9039f03ce9b8f1a6cd139bbf72c31c3b9df5ad15c6a98c2c0cc7b"} Jul 18 11:21:13.111759 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.111230 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-n5l4t" podStartSLOduration=2.19821724 podStartE2EDuration="4.111206014s" totalImagesPullingTime="1.912988774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:13.108638344 +0000 UTC m=+68.010402483" watchObservedRunningTime="2026-07-18 11:21:13.111206014 +0000 UTC m=+68.012970155" Jul 18 11:21:13.614639 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.614587 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:13.896618 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.895555 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f98fc96b-v85sn"] Jul 18 11:21:13.912985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.911919 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f98fc96b-v85sn"] Jul 18 11:21:13.912985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.912056 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:13.919205 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.918373 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 18 11:21:13.919205 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.918761 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 18 11:21:13.919205 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.918978 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 18 11:21:13.925295 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.924791 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 18 11:21:13.925295 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.925204 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9p8wv\"" Jul 18 11:21:13.925540 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:13.925452 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-co8j04e0cf42h\"" Jul 18 11:21:14.032718 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.032672 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d22b06b-8d48-4e69-9174-8428c1685dc4-audit-log\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.033086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.032733 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-client-ca-bundle\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.033086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.032771 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-secret-metrics-server-client-certs\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.033086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.032811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d22b06b-8d48-4e69-9174-8428c1685dc4-metrics-server-audit-profiles\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.033086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.032850 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-secret-metrics-server-tls\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.033086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.032900 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d22b06b-8d48-4e69-9174-8428c1685dc4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.033086 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.032931 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz685\" (UniqueName: \"kubernetes.io/projected/0d22b06b-8d48-4e69-9174-8428c1685dc4-kube-api-access-qz685\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.134325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.133875 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d22b06b-8d48-4e69-9174-8428c1685dc4-metrics-server-audit-profiles\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.134325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.133962 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-secret-metrics-server-tls\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.134325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.133998 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d22b06b-8d48-4e69-9174-8428c1685dc4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.134325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.134030 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz685\" (UniqueName: \"kubernetes.io/projected/0d22b06b-8d48-4e69-9174-8428c1685dc4-kube-api-access-qz685\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.134325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.134111 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d22b06b-8d48-4e69-9174-8428c1685dc4-audit-log\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.134325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.134153 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-client-ca-bundle\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.134325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.134188 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-secret-metrics-server-client-certs\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.135260 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.135084 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0d22b06b-8d48-4e69-9174-8428c1685dc4-metrics-server-audit-profiles\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.135260 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.135114 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0d22b06b-8d48-4e69-9174-8428c1685dc4-audit-log\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.135614 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.135572 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d22b06b-8d48-4e69-9174-8428c1685dc4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.138867 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.138802 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-client-ca-bundle\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.139075 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.139050 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-secret-metrics-server-tls\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.140227 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.140183 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0d22b06b-8d48-4e69-9174-8428c1685dc4-secret-metrics-server-client-certs\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.146538 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.146106 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz685\" (UniqueName: \"kubernetes.io/projected/0d22b06b-8d48-4e69-9174-8428c1685dc4-kube-api-access-qz685\") pod \"metrics-server-7f98fc96b-v85sn\" (UID: \"0d22b06b-8d48-4e69-9174-8428c1685dc4\") " pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.236525 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.234990 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:14.346874 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.346781 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:14.357205 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.357165 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826"] Jul 18 11:21:14.363209 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.362243 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:14.367207 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.365576 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-s6wqw\"" Jul 18 11:21:14.367207 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.365661 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 18 11:21:14.369714 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.369640 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826"] Jul 18 11:21:14.410059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.409906 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:14.438604 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.438391 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwdsv\" (UniqueName: \"kubernetes.io/projected/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-kube-api-access-mwdsv\") pod \"monitoring-plugin-7b7b7bc8bf-vg826\" (UID: \"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:14.438604 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.438516 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-vg826\" (UID: \"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:14.540714 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.539967 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-vg826\" (UID: \"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:14.540714 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.540037 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwdsv\" (UniqueName: \"kubernetes.io/projected/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-kube-api-access-mwdsv\") pod \"monitoring-plugin-7b7b7bc8bf-vg826\" (UID: \"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:14.540714 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:21:14.540461 2585 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 18 11:21:14.540714 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:21:14.540525 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-monitoring-plugin-cert podName:d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2 nodeName:}" failed. No retries permitted until 2026-07-18 11:21:15.040504243 +0000 UTC m=+69.942268362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-vg826" (UID: "d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2") : secret "monitoring-plugin-cert" not found Jul 18 11:21:14.554033 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.553962 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwdsv\" (UniqueName: \"kubernetes.io/projected/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-kube-api-access-mwdsv\") pod \"monitoring-plugin-7b7b7bc8bf-vg826\" (UID: \"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:14.859758 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:14.859664 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:15.045742 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:15.045198 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-vg826\" (UID: \"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:15.049721 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:15.049679 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-vg826\" (UID: \"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:15.280104 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:15.280065 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:15.880108 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:15.879820 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:17.039124 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:17.039014 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w6rlm" Jul 18 11:21:17.192641 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:17.192604 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:17.253475 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:17.253435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:17.515118 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:17.515078 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:17.581611 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:17.581573 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:17.804616 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:17.804518 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:17.855878 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:17.855837 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:19.691343 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:19.691307 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826"] Jul 18 11:21:20.616504 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:20.616459 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:20.716233 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:20.716078 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:22.051480 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:22.051435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:22.151899 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:22.151843 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:22.208188 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:22.208153 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:23.006747 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.006708 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f4b4fd5f6-8xxfc"] Jul 18 11:21:23.178539 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.178491 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4b4fd5f6-8xxfc"] Jul 18 11:21:23.178539 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.178530 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.181595 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.181542 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 18 11:21:23.181764 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.181607 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 18 11:21:23.181764 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.181607 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 18 11:21:23.182768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.182617 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 18 11:21:23.182768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.182635 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-t4pht\"" Jul 18 11:21:23.183123 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.183092 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 18 11:21:23.186811 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.186785 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 18 11:21:23.326476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.326376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-oauth-serving-cert\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.326476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.326439 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7jn2\" (UniqueName: \"kubernetes.io/projected/45be22c6-094c-4e22-bec0-3f564b932cb3-kube-api-access-g7jn2\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.326712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.326526 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-oauth-config\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.326712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.326557 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-service-ca\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.326712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.326612 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-serving-cert\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.326712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.326641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-console-config\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.326712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.326674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-trusted-ca-bundle\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.427620 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.427581 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-oauth-serving-cert\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.427810 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.427632 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7jn2\" (UniqueName: \"kubernetes.io/projected/45be22c6-094c-4e22-bec0-3f564b932cb3-kube-api-access-g7jn2\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.427810 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.427667 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-oauth-config\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.427810 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.427762 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-service-ca\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.427999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.427825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-serving-cert\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.427999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.427853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-console-config\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.427999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.427900 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-trusted-ca-bundle\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.428494 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.428455 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-service-ca\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.428494 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.428486 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-oauth-serving-cert\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.428691 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.428611 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-console-config\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.428791 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.428765 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-trusted-ca-bundle\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.430711 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.430684 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-oauth-config\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.430991 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.430971 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-serving-cert\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.437982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.437878 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7jn2\" (UniqueName: \"kubernetes.io/projected/45be22c6-094c-4e22-bec0-3f564b932cb3-kube-api-access-g7jn2\") pod \"console-6f4b4fd5f6-8xxfc\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:23.490817 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:23.490762 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:26.374827 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:26.374780 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:26.476832 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:26.476765 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:26.604697 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:26.604657 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:26.704786 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:26.704702 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:27.099175 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:27.099121 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f98fc96b-v85sn"] Jul 18 11:21:27.277093 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:27.277060 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c1cdb1_d0b2_49cf_a3dd_a53f041fdaa2.slice/crio-0875780a3b1b6e0a201c8dfc43af3fad6ff98de8bcf6f14d15ac2be14674092a WatchSource:0}: Error finding container 0875780a3b1b6e0a201c8dfc43af3fad6ff98de8bcf6f14d15ac2be14674092a: Status 404 returned error can't find the container with id 0875780a3b1b6e0a201c8dfc43af3fad6ff98de8bcf6f14d15ac2be14674092a Jul 18 11:21:27.425515 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:21:27.425482 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d22b06b_8d48_4e69_9174_8428c1685dc4.slice/crio-4659d398b7828297c11ba5087afb17da79cdf69d5fc523943227c87b9e9614db WatchSource:0}: Error finding container 4659d398b7828297c11ba5087afb17da79cdf69d5fc523943227c87b9e9614db: Status 404 returned error can't find the container with id 4659d398b7828297c11ba5087afb17da79cdf69d5fc523943227c87b9e9614db Jul 18 11:21:27.881559 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:27.881481 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4b4fd5f6-8xxfc"] Jul 18 11:21:28.172807 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.171551 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-mrw69" event={"ID":"0e141688-e460-49f5-87a9-44361c652b0b","Type":"ContainerStarted","Data":"368f68f70c12d7278560ebfdc8bd4ed8c8179f6f7aaa0e39b77b796623fd3076"} Jul 18 11:21:28.172807 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.172660 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-mrw69" Jul 18 11:21:28.179049 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.178633 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrfk" event={"ID":"a7154b70-e5dd-4f5a-b51a-41ad727aab16","Type":"ContainerStarted","Data":"47ffbbcd3f57b674cc454de636d52eebd06d58f64810fa37f222d67ab334726f"} Jul 18 11:21:28.179049 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.178691 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kxrfk" event={"ID":"a7154b70-e5dd-4f5a-b51a-41ad727aab16","Type":"ContainerStarted","Data":"82a3df13782e212cd59937816085150975b8025ae518c6d674d2785b30b7730f"} Jul 18 11:21:28.182420 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.181834 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-trnrr" event={"ID":"bb148327-e17d-470a-98ac-3790b9196f32","Type":"ContainerStarted","Data":"2d101096d4c078b0cc6ccf051cb4eddbcab04e81d39324b12355b95a5f95b129"} Jul 18 11:21:28.182420 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.182337 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:21:28.184406 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.184334 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4b4fd5f6-8xxfc" event={"ID":"45be22c6-094c-4e22-bec0-3f564b932cb3","Type":"ContainerStarted","Data":"4bc3b05808175807e78ac444adabd1011e79f01fd4d62b32bc16ca8820907271"} Jul 18 11:21:28.185601 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.185284 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-mrw69" Jul 18 11:21:28.189999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.189354 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" event={"ID":"686b7d6c-37f5-4082-8ed4-49b0d648fa18","Type":"ContainerStarted","Data":"816bfd403c8e728d584931e474f8687b36e76efae7be64bbcbfba3535f874b98"} Jul 18 11:21:28.189999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.189513 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" event={"ID":"686b7d6c-37f5-4082-8ed4-49b0d648fa18","Type":"ContainerStarted","Data":"2960069692c208d57e80b82fe1cd4a567daf6aed66cf9ef751e6d60f8cf3cc58"} Jul 18 11:21:28.198498 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.198189 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89wsg" event={"ID":"08c58c0e-cea2-4795-a53a-958964e776f4","Type":"ContainerStarted","Data":"2be85ab7307135d942247dc878aecf503d8b3dba7fa7568c2a4459d49e0c5fd7"} Jul 18 11:21:28.205139 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.205050 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" event={"ID":"0d22b06b-8d48-4e69-9174-8428c1685dc4","Type":"ContainerStarted","Data":"4659d398b7828297c11ba5087afb17da79cdf69d5fc523943227c87b9e9614db"} Jul 18 11:21:28.209908 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.209829 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" event={"ID":"f743c850-a5e0-4607-ba56-16bb19c1d0f8","Type":"ContainerStarted","Data":"df378a5e64f1d74d26cd0420c8fce0fa6e9640bdc80ef4ca7ea07d0d92ab2561"} Jul 18 11:21:28.212876 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.212757 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-mrw69" podStartSLOduration=2.997922951 podStartE2EDuration="25.212736805s" totalImagesPullingTime="22.214813854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:28.194602983 +0000 UTC m=+83.096367123" watchObservedRunningTime="2026-07-18 11:21:28.212736805 +0000 UTC m=+83.114500944" Jul 18 11:21:28.216422 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.216307 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8jxp7" event={"ID":"d1109b33-d650-4822-b3ab-08d1cd14d5a4","Type":"ContainerStarted","Data":"8e1d7d0cd091fa778acc2c7171822f789dcf9b83f42536545cf331bea00da651"} Jul 18 11:21:28.220562 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.220521 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx2rv" event={"ID":"ab36abb8-bb6e-458f-b909-277a53c37126","Type":"ContainerStarted","Data":"4b626ff111eb34a2cef4364b076af4d6ca1f007a3676f358ce40cdc05e04bf0a"} Jul 18 11:21:28.220743 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.220586 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx2rv" event={"ID":"ab36abb8-bb6e-458f-b909-277a53c37126","Type":"ContainerStarted","Data":"c689a09990363ef0650b92fb8bc6e9e19b4e8d6abc68d30023e820505892782d"} Jul 18 11:21:28.222705 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.222473 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" event={"ID":"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2","Type":"ContainerStarted","Data":"0875780a3b1b6e0a201c8dfc43af3fad6ff98de8bcf6f14d15ac2be14674092a"} Jul 18 11:21:28.236134 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.235165 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-trnrr" podStartSLOduration=66.816461296 podStartE2EDuration="1m22.235132281s" totalImagesPullingTime="15.418670985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:28.234071939 +0000 UTC m=+83.135836079" watchObservedRunningTime="2026-07-18 11:21:28.235132281 +0000 UTC m=+83.136896420" Jul 18 11:21:28.253482 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.253121 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kxrfk" podStartSLOduration=17.628995953 podStartE2EDuration="19.253099637s" totalImagesPullingTime="1.481216205s" totalInitContainerRuntime="142.887479ms" isStatefulPod=true podCreationTimestamp="2026-07-18 11:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:28.252339083 +0000 UTC m=+83.154103220" watchObservedRunningTime="2026-07-18 11:21:28.253099637 +0000 UTC m=+83.154863776" Jul 18 11:21:28.270942 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.270522 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8jxp7" podStartSLOduration=66.885266198 podStartE2EDuration="1m22.27050071s" totalImagesPullingTime="15.385234512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:28.269514358 +0000 UTC m=+83.171278497" watchObservedRunningTime="2026-07-18 11:21:28.27050071 +0000 UTC m=+83.172264850" Jul 18 11:21:28.344747 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.344672 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-89wsg" podStartSLOduration=3.307935378 podStartE2EDuration="30.344646317s" totalImagesPullingTime="27.036710939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:20:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:28.293953031 +0000 UTC m=+83.195717170" watchObservedRunningTime="2026-07-18 11:21:28.344646317 +0000 UTC m=+83.246410458" Jul 18 11:21:28.367706 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.365972 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-z2tpm" podStartSLOduration=10.663892345 podStartE2EDuration="19.365945872s" totalImagesPullingTime="8.702053527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:28.34529131 +0000 UTC m=+83.247055448" watchObservedRunningTime="2026-07-18 11:21:28.365945872 +0000 UTC m=+83.267710012" Jul 18 11:21:28.367706 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.367276 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bx2rv" podStartSLOduration=66.929595722 podStartE2EDuration="1m22.367262137s" totalImagesPullingTime="15.437666415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:28.366451836 +0000 UTC m=+83.268215987" watchObservedRunningTime="2026-07-18 11:21:28.367262137 +0000 UTC m=+83.269026277" Jul 18 11:21:28.452683 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.452516 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:28.562059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.561992 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:28.654473 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.654432 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:28.817440 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.817396 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:28.920118 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:28.920060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:29.235572 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:29.235220 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" event={"ID":"686b7d6c-37f5-4082-8ed4-49b0d648fa18","Type":"ContainerStarted","Data":"a670cd28c16802c4b8eb1356760a07f81e0a56c52ed254c776d14bf67298cf6d"} Jul 18 11:21:31.198316 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:31.197684 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:31.300676 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:31.300608 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:31.411721 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:31.411683 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:31.512387 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:31.512313 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:33.263909 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.263840 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" event={"ID":"0d22b06b-8d48-4e69-9174-8428c1685dc4","Type":"ContainerStarted","Data":"3fb60cec751d6e83540d5467ea3f4491f7e973cd74c790936cffad159b8de05d"} Jul 18 11:21:33.266237 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.266201 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" event={"ID":"d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2","Type":"ContainerStarted","Data":"9d1c66dae70e025b440fb0d782a4b8b406ce0a6a979918a8e7cda5318dc97d18"} Jul 18 11:21:33.266650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.266614 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:33.268688 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.268632 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4b4fd5f6-8xxfc" event={"ID":"45be22c6-094c-4e22-bec0-3f564b932cb3","Type":"ContainerStarted","Data":"b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b"} Jul 18 11:21:33.272994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.272942 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" event={"ID":"686b7d6c-37f5-4082-8ed4-49b0d648fa18","Type":"ContainerStarted","Data":"d4ddda8ccf900151059077989054ff63ee2ae38f4eb351134b88d0aaf7826441"} Jul 18 11:21:33.273166 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.273039 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" event={"ID":"686b7d6c-37f5-4082-8ed4-49b0d648fa18","Type":"ContainerStarted","Data":"083a37e20567de4e94779b61de44b1ef538e469a6c1515c43259202addb8f3b2"} Jul 18 11:21:33.273166 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.273081 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" event={"ID":"686b7d6c-37f5-4082-8ed4-49b0d648fa18","Type":"ContainerStarted","Data":"ab38780d76138828086ea532f2799bfe9585c99e3ca3eeecdbf3917655ba661f"} Jul 18 11:21:33.273166 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.273121 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:33.274342 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.274318 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" Jul 18 11:21:33.279852 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.279824 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" Jul 18 11:21:33.297599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.297529 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" podStartSLOduration=15.35020713 podStartE2EDuration="20.297507437s" totalImagesPullingTime="4.947300307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:33.282099671 +0000 UTC m=+88.183863809" watchObservedRunningTime="2026-07-18 11:21:33.297507437 +0000 UTC m=+88.199271581" Jul 18 11:21:33.297911 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.297859 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-vg826" podStartSLOduration=14.329966301 podStartE2EDuration="19.297850387s" totalImagesPullingTime="4.967884086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:33.296311279 +0000 UTC m=+88.198075419" watchObservedRunningTime="2026-07-18 11:21:33.297850387 +0000 UTC m=+88.199614502" Jul 18 11:21:33.323822 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.322772 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65f7c6c966-wqwxs" podStartSLOduration=2.518178477 podStartE2EDuration="22.322747214s" totalImagesPullingTime="19.804568737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:33.319628594 +0000 UTC m=+88.221392733" watchObservedRunningTime="2026-07-18 11:21:33.322747214 +0000 UTC m=+88.224511353" Jul 18 11:21:33.338731 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.338349 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f4b4fd5f6-8xxfc" podStartSLOduration=6.620027568 podStartE2EDuration="11.338325647s" totalImagesPullingTime="4.718298079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:21:33.336317021 +0000 UTC m=+88.238081162" watchObservedRunningTime="2026-07-18 11:21:33.338325647 +0000 UTC m=+88.240089786" Jul 18 11:21:33.491506 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.491462 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:33.491730 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.491532 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:33.497277 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:33.497236 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:34.236178 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:34.236137 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:34.236468 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:34.236220 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:34.285076 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:34.285036 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:21:34.637839 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:34.637794 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:41.607007 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:41.606961 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:41.706611 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:41.706573 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:21:54.241594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:54.241558 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:54.245483 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:54.245451 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f98fc96b-v85sn" Jul 18 11:21:59.239349 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:21:59.239316 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-trnrr" Jul 18 11:22:28.814779 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:28.814741 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:29.132242 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:29.132148 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:33.289374 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.289335 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:33.891252 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.891204 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69b6fbf784-t68rd"] Jul 18 11:22:33.894155 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.894132 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:33.900034 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.900002 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 18 11:22:33.900443 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.900401 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 18 11:22:33.901749 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.901713 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 18 11:22:33.901871 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.901790 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 18 11:22:33.901961 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.901947 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 18 11:22:33.902454 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.902432 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zg8wm\"" Jul 18 11:22:33.907080 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.907022 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 18 11:22:33.924098 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:33.924066 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b6fbf784-t68rd"] Jul 18 11:22:34.029503 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029441 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-telemeter-client-tls\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.029503 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.029780 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029538 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-metrics-client-ca\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.029780 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029569 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-secret-telemeter-client\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.029780 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.029780 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029716 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-serving-certs-ca-bundle\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.029780 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-federate-client-tls\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.029780 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.029755 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/4d310666-ad0f-44b4-a218-9e13367fd168-kube-api-access-7v2q4\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130479 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130479 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-serving-certs-ca-bundle\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130744 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130505 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-federate-client-tls\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130744 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130521 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/4d310666-ad0f-44b4-a218-9e13367fd168-kube-api-access-7v2q4\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130744 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130545 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-telemeter-client-tls\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130744 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130582 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130979 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130758 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-metrics-client-ca\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.130979 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.130823 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-secret-telemeter-client\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.131462 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.131428 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-serving-certs-ca-bundle\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.131590 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.131488 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-metrics-client-ca\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.131648 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.131593 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d310666-ad0f-44b4-a218-9e13367fd168-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.135398 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.135366 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-secret-telemeter-client\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.135529 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.135400 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.135594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.135428 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-telemeter-client-tls\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.135594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.135464 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d310666-ad0f-44b4-a218-9e13367fd168-federate-client-tls\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.140589 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.140558 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/4d310666-ad0f-44b4-a218-9e13367fd168-kube-api-access-7v2q4\") pod \"telemeter-client-69b6fbf784-t68rd\" (UID: \"4d310666-ad0f-44b4-a218-9e13367fd168\") " pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.210942 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.210826 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" Jul 18 11:22:34.385139 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.385107 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b6fbf784-t68rd"] Jul 18 11:22:34.387666 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:22:34.387631 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d310666_ad0f_44b4_a218_9e13367fd168.slice/crio-f1b907537b5279799213d54bcad0c5c5fb5af730175152d9dad6d9dfc42430e3 WatchSource:0}: Error finding container f1b907537b5279799213d54bcad0c5c5fb5af730175152d9dad6d9dfc42430e3: Status 404 returned error can't find the container with id f1b907537b5279799213d54bcad0c5c5fb5af730175152d9dad6d9dfc42430e3 Jul 18 11:22:34.505445 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:34.505397 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" event={"ID":"4d310666-ad0f-44b4-a218-9e13367fd168","Type":"ContainerStarted","Data":"f1b907537b5279799213d54bcad0c5c5fb5af730175152d9dad6d9dfc42430e3"} Jul 18 11:22:36.515650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:36.515554 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" event={"ID":"4d310666-ad0f-44b4-a218-9e13367fd168","Type":"ContainerStarted","Data":"5769684adf21f90ea3526fa22bcb9bb21ed51b4d0de33d1acdfcd0298b62a2d6"} Jul 18 11:22:37.522531 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:37.522429 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" event={"ID":"4d310666-ad0f-44b4-a218-9e13367fd168","Type":"ContainerStarted","Data":"8025981d8a32c35ba3381a6aa0993ded690f7e9a935b0e3bc32cfe08b34f263a"} Jul 18 11:22:37.522531 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:37.522503 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" event={"ID":"4d310666-ad0f-44b4-a218-9e13367fd168","Type":"ContainerStarted","Data":"57425c2fa0c644421be00a828ab2c522a367c072607f38154098fef2e0703bf7"} Jul 18 11:22:37.547551 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:37.547498 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69b6fbf784-t68rd" podStartSLOduration=1.740643658 podStartE2EDuration="4.547477461s" totalImagesPullingTime="2.806833803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:22:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:22:37.545306972 +0000 UTC m=+152.447071110" watchObservedRunningTime="2026-07-18 11:22:37.547477461 +0000 UTC m=+152.449241598" Jul 18 11:22:38.302957 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.302915 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b54fd68f8-9tf9x"] Jul 18 11:22:38.305604 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.305574 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.317560 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.317527 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b54fd68f8-9tf9x"] Jul 18 11:22:38.365793 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.365749 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:38.366837 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.366813 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-serving-cert\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.366960 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.366863 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qc78\" (UniqueName: \"kubernetes.io/projected/168c15c6-2af9-4769-b79f-2b05f5890525-kube-api-access-4qc78\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.367000 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.366949 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-service-ca\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.367000 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.366987 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-trusted-ca-bundle\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.367067 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.367007 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-console-config\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.367067 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.367027 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-oauth-serving-cert\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.367067 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.367052 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-oauth-config\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.420612 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.420571 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:38.467599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.467562 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-oauth-serving-cert\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.467599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.467599 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-oauth-config\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.467845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.467631 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-serving-cert\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.467845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.467669 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qc78\" (UniqueName: \"kubernetes.io/projected/168c15c6-2af9-4769-b79f-2b05f5890525-kube-api-access-4qc78\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.467845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.467690 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-service-ca\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.467845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.467716 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-trusted-ca-bundle\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.467845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.467742 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-console-config\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.468437 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.468403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-oauth-serving-cert\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.468554 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.468403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-service-ca\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.468626 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.468609 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-console-config\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.468706 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.468687 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-trusted-ca-bundle\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.470704 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.470681 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-oauth-config\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.470992 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.470972 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-serving-cert\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.475616 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.475593 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qc78\" (UniqueName: \"kubernetes.io/projected/168c15c6-2af9-4769-b79f-2b05f5890525-kube-api-access-4qc78\") pod \"console-5b54fd68f8-9tf9x\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.569629 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.569528 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:38.617418 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.617380 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:38.668520 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.668486 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:38.749588 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:38.749485 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b54fd68f8-9tf9x"] Jul 18 11:22:38.752367 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:22:38.752341 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168c15c6_2af9_4769_b79f_2b05f5890525.slice/crio-fc9ba2f8e1277f2ac61addf5edc722d0f0a166d529a528fa01527e1d744f0a11 WatchSource:0}: Error finding container fc9ba2f8e1277f2ac61addf5edc722d0f0a166d529a528fa01527e1d744f0a11: Status 404 returned error can't find the container with id fc9ba2f8e1277f2ac61addf5edc722d0f0a166d529a528fa01527e1d744f0a11 Jul 18 11:22:39.098478 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:39.098430 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:39.158191 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:39.158156 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:39.532307 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:39.532260 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54fd68f8-9tf9x" event={"ID":"168c15c6-2af9-4769-b79f-2b05f5890525","Type":"ContainerStarted","Data":"102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7"} Jul 18 11:22:39.532485 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:39.532322 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54fd68f8-9tf9x" event={"ID":"168c15c6-2af9-4769-b79f-2b05f5890525","Type":"ContainerStarted","Data":"fc9ba2f8e1277f2ac61addf5edc722d0f0a166d529a528fa01527e1d744f0a11"} Jul 18 11:22:39.549794 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:39.549744 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b54fd68f8-9tf9x" podStartSLOduration=1.5497285330000001 podStartE2EDuration="1.549728533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:22:39.549486058 +0000 UTC m=+154.451250198" watchObservedRunningTime="2026-07-18 11:22:39.549728533 +0000 UTC m=+154.451492672" Jul 18 11:22:48.617914 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:48.617849 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:48.618300 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:48.617962 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:48.622595 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:48.622556 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:49.574000 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:49.573971 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:22:49.619369 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:49.619336 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4b4fd5f6-8xxfc"] Jul 18 11:22:49.720462 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:49.720426 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:49.821493 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:49.821458 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:22:49.870606 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:22:49.870511 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:14.645096 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.645035 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f4b4fd5f6-8xxfc" podUID="45be22c6-094c-4e22-bec0-3f564b932cb3" containerName="console" containerID="cri-o://b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b" gracePeriod=15 Jul 18 11:23:14.888274 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.888247 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:23:14.953416 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.953317 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-serving-cert\") pod \"45be22c6-094c-4e22-bec0-3f564b932cb3\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " Jul 18 11:23:14.953416 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.953354 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-oauth-config\") pod \"45be22c6-094c-4e22-bec0-3f564b932cb3\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " Jul 18 11:23:14.953416 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.953404 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-service-ca\") pod \"45be22c6-094c-4e22-bec0-3f564b932cb3\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " Jul 18 11:23:14.953697 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.953448 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-console-config\") pod \"45be22c6-094c-4e22-bec0-3f564b932cb3\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " Jul 18 11:23:14.953697 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.953481 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-oauth-serving-cert\") pod \"45be22c6-094c-4e22-bec0-3f564b932cb3\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " Jul 18 11:23:14.953697 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.953512 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7jn2\" (UniqueName: \"kubernetes.io/projected/45be22c6-094c-4e22-bec0-3f564b932cb3-kube-api-access-g7jn2\") pod \"45be22c6-094c-4e22-bec0-3f564b932cb3\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " Jul 18 11:23:14.953697 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.953535 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-trusted-ca-bundle\") pod \"45be22c6-094c-4e22-bec0-3f564b932cb3\" (UID: \"45be22c6-094c-4e22-bec0-3f564b932cb3\") " Jul 18 11:23:14.954050 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.954003 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-service-ca" (OuterVolumeSpecName: "service-ca") pod "45be22c6-094c-4e22-bec0-3f564b932cb3" (UID: "45be22c6-094c-4e22-bec0-3f564b932cb3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:23:14.954050 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.954042 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "45be22c6-094c-4e22-bec0-3f564b932cb3" (UID: "45be22c6-094c-4e22-bec0-3f564b932cb3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:23:14.954226 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.954011 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-console-config" (OuterVolumeSpecName: "console-config") pod "45be22c6-094c-4e22-bec0-3f564b932cb3" (UID: "45be22c6-094c-4e22-bec0-3f564b932cb3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:23:14.954226 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.954075 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "45be22c6-094c-4e22-bec0-3f564b932cb3" (UID: "45be22c6-094c-4e22-bec0-3f564b932cb3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:23:14.955818 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.955789 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "45be22c6-094c-4e22-bec0-3f564b932cb3" (UID: "45be22c6-094c-4e22-bec0-3f564b932cb3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:23:14.956077 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.956047 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "45be22c6-094c-4e22-bec0-3f564b932cb3" (UID: "45be22c6-094c-4e22-bec0-3f564b932cb3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:23:14.956077 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:14.956052 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45be22c6-094c-4e22-bec0-3f564b932cb3-kube-api-access-g7jn2" (OuterVolumeSpecName: "kube-api-access-g7jn2") pod "45be22c6-094c-4e22-bec0-3f564b932cb3" (UID: "45be22c6-094c-4e22-bec0-3f564b932cb3"). InnerVolumeSpecName "kube-api-access-g7jn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:23:15.055092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.055057 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-serving-cert\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:23:15.055092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.055085 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45be22c6-094c-4e22-bec0-3f564b932cb3-console-oauth-config\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:23:15.055092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.055095 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:23:15.055324 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.055105 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-console-config\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:23:15.055324 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.055113 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-oauth-serving-cert\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:23:15.055324 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.055121 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7jn2\" (UniqueName: \"kubernetes.io/projected/45be22c6-094c-4e22-bec0-3f564b932cb3-kube-api-access-g7jn2\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:23:15.055324 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.055130 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45be22c6-094c-4e22-bec0-3f564b932cb3-trusted-ca-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:23:15.667384 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.667340 2585 generic.go:356] "Generic (PLEG): container finished" podID="45be22c6-094c-4e22-bec0-3f564b932cb3" containerID="b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b" exitCode=0 Jul 18 11:23:15.667384 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.667389 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4b4fd5f6-8xxfc" Jul 18 11:23:15.667844 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.667424 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4b4fd5f6-8xxfc" event={"ID":"45be22c6-094c-4e22-bec0-3f564b932cb3","Type":"ContainerDied","Data":"b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b"} Jul 18 11:23:15.667844 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.667481 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4b4fd5f6-8xxfc" event={"ID":"45be22c6-094c-4e22-bec0-3f564b932cb3","Type":"ContainerDied","Data":"4bc3b05808175807e78ac444adabd1011e79f01fd4d62b32bc16ca8820907271"} Jul 18 11:23:15.667844 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.667504 2585 scope.go:117] "RemoveContainer" containerID="b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b" Jul 18 11:23:15.676206 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.676185 2585 scope.go:117] "RemoveContainer" containerID="b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b" Jul 18 11:23:15.676609 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:23:15.676585 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b\": container with ID starting with b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b not found: ID does not exist" containerID="b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b" Jul 18 11:23:15.676695 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.676620 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b"} err="failed to get container status \"b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b\": rpc error: code = NotFound desc = could not find container \"b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b\": container with ID starting with b30f74cfe772a9f51ee6c5b32b7fbef797d7e218b42298627e900d464e833d5b not found: ID does not exist" Jul 18 11:23:15.684790 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.684754 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4b4fd5f6-8xxfc"] Jul 18 11:23:15.688491 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:15.688460 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f4b4fd5f6-8xxfc"] Jul 18 11:23:17.613555 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:17.613509 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45be22c6-094c-4e22-bec0-3f564b932cb3" path="/var/lib/kubelet/pods/45be22c6-094c-4e22-bec0-3f564b932cb3/volumes" Jul 18 11:23:34.378821 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:34.378779 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:44.058735 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:44.058700 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:44.115535 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:44.115495 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:45.033925 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.033873 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b766c647-2snv8"] Jul 18 11:23:45.034241 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.034226 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45be22c6-094c-4e22-bec0-3f564b932cb3" containerName="console" Jul 18 11:23:45.034241 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.034242 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="45be22c6-094c-4e22-bec0-3f564b932cb3" containerName="console" Jul 18 11:23:45.034356 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.034296 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="45be22c6-094c-4e22-bec0-3f564b932cb3" containerName="console" Jul 18 11:23:45.037222 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.037201 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.049478 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.049444 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b766c647-2snv8"] Jul 18 11:23:45.082336 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.082298 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:45.134769 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.134725 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:45.205229 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.205190 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-console-config\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.205391 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.205239 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb9b4\" (UniqueName: \"kubernetes.io/projected/079d31bb-d538-48be-a459-f0821e675178-kube-api-access-wb9b4\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.205391 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.205263 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-oauth-serving-cert\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.205391 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.205333 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-oauth-config\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.205391 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.205369 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-trusted-ca-bundle\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.205520 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.205401 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-service-ca\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.205520 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.205423 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-serving-cert\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.306611 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.306516 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb9b4\" (UniqueName: \"kubernetes.io/projected/079d31bb-d538-48be-a459-f0821e675178-kube-api-access-wb9b4\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.306611 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.306559 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-oauth-serving-cert\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.306611 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.306589 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-oauth-config\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.306932 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.306621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-trusted-ca-bundle\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.306932 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.306654 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-service-ca\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.306932 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.306680 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-serving-cert\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.306932 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.306724 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-console-config\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.307475 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.307446 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-oauth-serving-cert\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.307599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.307469 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-service-ca\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.307599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.307447 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-console-config\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.307599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.307529 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-trusted-ca-bundle\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.309397 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.309372 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-oauth-config\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.309480 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.309394 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-serving-cert\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.315147 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.315108 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb9b4\" (UniqueName: \"kubernetes.io/projected/079d31bb-d538-48be-a459-f0821e675178-kube-api-access-wb9b4\") pod \"console-77b766c647-2snv8\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.345720 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.345683 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:45.482350 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.482320 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b766c647-2snv8"] Jul 18 11:23:45.485335 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:23:45.485307 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079d31bb_d538_48be_a459_f0821e675178.slice/crio-958959b5f8849c7b580d6a110a9b98c46aa7166c7b85be89737c331251c06920 WatchSource:0}: Error finding container 958959b5f8849c7b580d6a110a9b98c46aa7166c7b85be89737c331251c06920: Status 404 returned error can't find the container with id 958959b5f8849c7b580d6a110a9b98c46aa7166c7b85be89737c331251c06920 Jul 18 11:23:45.792121 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.792082 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b766c647-2snv8" event={"ID":"079d31bb-d538-48be-a459-f0821e675178","Type":"ContainerStarted","Data":"b9389c64e994b206f87a1608db0ef44aaab5ff659df36e7ff52fac16fb4eabed"} Jul 18 11:23:45.792321 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.792168 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b766c647-2snv8" event={"ID":"079d31bb-d538-48be-a459-f0821e675178","Type":"ContainerStarted","Data":"958959b5f8849c7b580d6a110a9b98c46aa7166c7b85be89737c331251c06920"} Jul 18 11:23:45.809553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:45.809495 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b766c647-2snv8" podStartSLOduration=0.809477302 podStartE2EDuration="809.477302ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:23:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:23:45.808958491 +0000 UTC m=+220.710722631" watchObservedRunningTime="2026-07-18 11:23:45.809477302 +0000 UTC m=+220.711241516" Jul 18 11:23:55.346150 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:55.346116 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:55.346538 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:55.346176 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:55.350820 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:55.350797 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:55.834533 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:55.834505 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:23:55.876693 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:55.876660 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b54fd68f8-9tf9x"] Jul 18 11:23:55.982238 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:55.982197 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:56.080695 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:56.080661 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:23:56.132435 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:23:56.132354 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:24:11.617566 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:11.617528 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:24:11.719555 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:11.719519 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:24:20.898457 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:20.898412 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b54fd68f8-9tf9x" podUID="168c15c6-2af9-4769-b79f-2b05f5890525" containerName="console" containerID="cri-o://102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7" gracePeriod=15 Jul 18 11:24:21.138265 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.138231 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:24:21.174913 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.174811 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-service-ca\") pod \"168c15c6-2af9-4769-b79f-2b05f5890525\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " Jul 18 11:24:21.175059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.174914 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-trusted-ca-bundle\") pod \"168c15c6-2af9-4769-b79f-2b05f5890525\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " Jul 18 11:24:21.175059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.174947 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-console-config\") pod \"168c15c6-2af9-4769-b79f-2b05f5890525\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " Jul 18 11:24:21.175059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.174978 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-oauth-config\") pod \"168c15c6-2af9-4769-b79f-2b05f5890525\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " Jul 18 11:24:21.175059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.175008 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-oauth-serving-cert\") pod \"168c15c6-2af9-4769-b79f-2b05f5890525\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " Jul 18 11:24:21.175059 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.175035 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-serving-cert\") pod \"168c15c6-2af9-4769-b79f-2b05f5890525\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " Jul 18 11:24:21.175305 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.175083 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qc78\" (UniqueName: \"kubernetes.io/projected/168c15c6-2af9-4769-b79f-2b05f5890525-kube-api-access-4qc78\") pod \"168c15c6-2af9-4769-b79f-2b05f5890525\" (UID: \"168c15c6-2af9-4769-b79f-2b05f5890525\") " Jul 18 11:24:21.175370 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.175313 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-service-ca" (OuterVolumeSpecName: "service-ca") pod "168c15c6-2af9-4769-b79f-2b05f5890525" (UID: "168c15c6-2af9-4769-b79f-2b05f5890525"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:24:21.175527 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.175503 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "168c15c6-2af9-4769-b79f-2b05f5890525" (UID: "168c15c6-2af9-4769-b79f-2b05f5890525"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:24:21.175594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.175531 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-console-config" (OuterVolumeSpecName: "console-config") pod "168c15c6-2af9-4769-b79f-2b05f5890525" (UID: "168c15c6-2af9-4769-b79f-2b05f5890525"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:24:21.175594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.175538 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "168c15c6-2af9-4769-b79f-2b05f5890525" (UID: "168c15c6-2af9-4769-b79f-2b05f5890525"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:24:21.177255 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.177219 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "168c15c6-2af9-4769-b79f-2b05f5890525" (UID: "168c15c6-2af9-4769-b79f-2b05f5890525"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:24:21.177255 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.177247 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "168c15c6-2af9-4769-b79f-2b05f5890525" (UID: "168c15c6-2af9-4769-b79f-2b05f5890525"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:24:21.177421 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.177279 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/168c15c6-2af9-4769-b79f-2b05f5890525-kube-api-access-4qc78" (OuterVolumeSpecName: "kube-api-access-4qc78") pod "168c15c6-2af9-4769-b79f-2b05f5890525" (UID: "168c15c6-2af9-4769-b79f-2b05f5890525"). InnerVolumeSpecName "kube-api-access-4qc78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:24:21.275819 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.275769 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:24:21.275819 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.275814 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-trusted-ca-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:24:21.275819 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.275826 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-console-config\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:24:21.275819 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.275836 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-oauth-config\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:24:21.276123 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.275846 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/168c15c6-2af9-4769-b79f-2b05f5890525-oauth-serving-cert\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:24:21.276123 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.275856 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/168c15c6-2af9-4769-b79f-2b05f5890525-console-serving-cert\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:24:21.276123 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.275864 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qc78\" (UniqueName: \"kubernetes.io/projected/168c15c6-2af9-4769-b79f-2b05f5890525-kube-api-access-4qc78\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:24:21.933359 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.933271 2585 generic.go:356] "Generic (PLEG): container finished" podID="168c15c6-2af9-4769-b79f-2b05f5890525" containerID="102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7" exitCode=0 Jul 18 11:24:21.933359 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.933312 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54fd68f8-9tf9x" Jul 18 11:24:21.933778 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.933368 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54fd68f8-9tf9x" event={"ID":"168c15c6-2af9-4769-b79f-2b05f5890525","Type":"ContainerDied","Data":"102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7"} Jul 18 11:24:21.933778 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.933426 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54fd68f8-9tf9x" event={"ID":"168c15c6-2af9-4769-b79f-2b05f5890525","Type":"ContainerDied","Data":"fc9ba2f8e1277f2ac61addf5edc722d0f0a166d529a528fa01527e1d744f0a11"} Jul 18 11:24:21.933778 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.933451 2585 scope.go:117] "RemoveContainer" containerID="102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7" Jul 18 11:24:21.942037 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.942017 2585 scope.go:117] "RemoveContainer" containerID="102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7" Jul 18 11:24:21.942341 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:24:21.942318 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7\": container with ID starting with 102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7 not found: ID does not exist" containerID="102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7" Jul 18 11:24:21.942409 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.942355 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7"} err="failed to get container status \"102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7\": rpc error: code = NotFound desc = could not find container \"102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7\": container with ID starting with 102b229e848435c24cdcb7e3d08637eb67dc77c86f38e9c1a122f7adbfbb70e7 not found: ID does not exist" Jul 18 11:24:21.950505 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.950474 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b54fd68f8-9tf9x"] Jul 18 11:24:21.956739 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:21.956709 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b54fd68f8-9tf9x"] Jul 18 11:24:23.613411 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:24:23.613373 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="168c15c6-2af9-4769-b79f-2b05f5890525" path="/var/lib/kubelet/pods/168c15c6-2af9-4769-b79f-2b05f5890525/volumes" Jul 18 11:25:05.482954 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:05.477915 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 11:25:07.702510 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:07.702452 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:08.293379 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.293340 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94"] Jul 18 11:25:08.293716 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.293699 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168c15c6-2af9-4769-b79f-2b05f5890525" containerName="console" Jul 18 11:25:08.293802 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.293719 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="168c15c6-2af9-4769-b79f-2b05f5890525" containerName="console" Jul 18 11:25:08.293857 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.293818 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="168c15c6-2af9-4769-b79f-2b05f5890525" containerName="console" Jul 18 11:25:08.297003 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.296974 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.300013 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.299981 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 11:25:08.300013 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.300011 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9tnk2\"" Jul 18 11:25:08.300199 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.299981 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 11:25:08.304912 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.304870 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94"] Jul 18 11:25:08.361005 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.360962 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.361005 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.361005 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.361249 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.361082 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qv8r\" (UniqueName: \"kubernetes.io/projected/990de817-0ef0-46af-a661-b5e6e037ebd8-kube-api-access-4qv8r\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.462411 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.462366 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.462411 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.462411 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.462626 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.462452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qv8r\" (UniqueName: \"kubernetes.io/projected/990de817-0ef0-46af-a661-b5e6e037ebd8-kube-api-access-4qv8r\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.462785 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.462766 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.462848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.462824 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.471644 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.471612 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qv8r\" (UniqueName: \"kubernetes.io/projected/990de817-0ef0-46af-a661-b5e6e037ebd8-kube-api-access-4qv8r\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.607605 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.607516 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:08.704538 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.704480 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:08.745264 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.745114 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94"] Jul 18 11:25:08.748067 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:25:08.748036 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990de817_0ef0_46af_a661_b5e6e037ebd8.slice/crio-015a3cf4428d7b4b50d0c5b739c0a20242972e16b8cbbb39630a5e79e3e362e7 WatchSource:0}: Error finding container 015a3cf4428d7b4b50d0c5b739c0a20242972e16b8cbbb39630a5e79e3e362e7: Status 404 returned error can't find the container with id 015a3cf4428d7b4b50d0c5b739c0a20242972e16b8cbbb39630a5e79e3e362e7 Jul 18 11:25:08.750075 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:08.750057 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:25:09.114754 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:09.114715 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" event={"ID":"990de817-0ef0-46af-a661-b5e6e037ebd8","Type":"ContainerStarted","Data":"015a3cf4428d7b4b50d0c5b739c0a20242972e16b8cbbb39630a5e79e3e362e7"} Jul 18 11:25:16.147460 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:16.147423 2585 generic.go:356] "Generic (PLEG): container finished" podID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerID="6365c47586ef3478d88e5589b294ac2ba68aa0524a7dfd5920086ce72d096c72" exitCode=0 Jul 18 11:25:16.147841 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:16.147515 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" event={"ID":"990de817-0ef0-46af-a661-b5e6e037ebd8","Type":"ContainerDied","Data":"6365c47586ef3478d88e5589b294ac2ba68aa0524a7dfd5920086ce72d096c72"} Jul 18 11:25:19.163136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:19.163096 2585 generic.go:356] "Generic (PLEG): container finished" podID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerID="b223f5a6e0a73648eec9151a093b030337fd31cd8139de0fb912a494f50f98a3" exitCode=0 Jul 18 11:25:19.163600 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:19.163184 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" event={"ID":"990de817-0ef0-46af-a661-b5e6e037ebd8","Type":"ContainerDied","Data":"b223f5a6e0a73648eec9151a093b030337fd31cd8139de0fb912a494f50f98a3"} Jul 18 11:25:26.196351 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:26.196311 2585 generic.go:356] "Generic (PLEG): container finished" podID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerID="993251b2049ca0d4f207dab0b7ef5567033502c0224d50c4ce79894943991ea5" exitCode=0 Jul 18 11:25:26.196794 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:26.196368 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" event={"ID":"990de817-0ef0-46af-a661-b5e6e037ebd8","Type":"ContainerDied","Data":"993251b2049ca0d4f207dab0b7ef5567033502c0224d50c4ce79894943991ea5"} Jul 18 11:25:27.339784 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.339743 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:27.428211 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.428172 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qv8r\" (UniqueName: \"kubernetes.io/projected/990de817-0ef0-46af-a661-b5e6e037ebd8-kube-api-access-4qv8r\") pod \"990de817-0ef0-46af-a661-b5e6e037ebd8\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " Jul 18 11:25:27.428382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.428223 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-util\") pod \"990de817-0ef0-46af-a661-b5e6e037ebd8\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " Jul 18 11:25:27.428382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.428288 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-bundle\") pod \"990de817-0ef0-46af-a661-b5e6e037ebd8\" (UID: \"990de817-0ef0-46af-a661-b5e6e037ebd8\") " Jul 18 11:25:27.429020 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.428991 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-bundle" (OuterVolumeSpecName: "bundle") pod "990de817-0ef0-46af-a661-b5e6e037ebd8" (UID: "990de817-0ef0-46af-a661-b5e6e037ebd8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:25:27.430528 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.430499 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/990de817-0ef0-46af-a661-b5e6e037ebd8-kube-api-access-4qv8r" (OuterVolumeSpecName: "kube-api-access-4qv8r") pod "990de817-0ef0-46af-a661-b5e6e037ebd8" (UID: "990de817-0ef0-46af-a661-b5e6e037ebd8"). InnerVolumeSpecName "kube-api-access-4qv8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:25:27.432515 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.432487 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-util" (OuterVolumeSpecName: "util") pod "990de817-0ef0-46af-a661-b5e6e037ebd8" (UID: "990de817-0ef0-46af-a661-b5e6e037ebd8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:25:27.529775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.529748 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:25:27.529775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.529775 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qv8r\" (UniqueName: \"kubernetes.io/projected/990de817-0ef0-46af-a661-b5e6e037ebd8-kube-api-access-4qv8r\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:25:27.529993 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:27.529789 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/990de817-0ef0-46af-a661-b5e6e037ebd8-util\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:25:28.205845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:28.205816 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" Jul 18 11:25:28.206022 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:28.205811 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540snx94" event={"ID":"990de817-0ef0-46af-a661-b5e6e037ebd8","Type":"ContainerDied","Data":"015a3cf4428d7b4b50d0c5b739c0a20242972e16b8cbbb39630a5e79e3e362e7"} Jul 18 11:25:28.206022 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:28.205930 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="015a3cf4428d7b4b50d0c5b739c0a20242972e16b8cbbb39630a5e79e3e362e7" Jul 18 11:25:29.278444 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:29.278411 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:30.077254 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:30.077213 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:31.192501 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192464 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf"] Jul 18 11:25:31.192916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192764 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerName="pull" Jul 18 11:25:31.192916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192775 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerName="pull" Jul 18 11:25:31.192916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192781 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerName="extract" Jul 18 11:25:31.192916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192787 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerName="extract" Jul 18 11:25:31.192916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192805 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerName="util" Jul 18 11:25:31.192916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192811 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerName="util" Jul 18 11:25:31.192916 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.192861 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="990de817-0ef0-46af-a661-b5e6e037ebd8" containerName="extract" Jul 18 11:25:31.194563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.194547 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.197384 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.197357 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 18 11:25:31.197569 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.197550 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:25:31.197657 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.197588 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4q97f\"" Jul 18 11:25:31.208191 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.208160 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf"] Jul 18 11:25:31.261987 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.261947 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bdcba16-fb70-4578-b415-23549724b425-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ddstf\" (UID: \"8bdcba16-fb70-4578-b415-23549724b425\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.262150 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.262001 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mshjr\" (UniqueName: \"kubernetes.io/projected/8bdcba16-fb70-4578-b415-23549724b425-kube-api-access-mshjr\") pod \"cert-manager-operator-controller-manager-55c444bf86-ddstf\" (UID: \"8bdcba16-fb70-4578-b415-23549724b425\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.362936 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.362869 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mshjr\" (UniqueName: \"kubernetes.io/projected/8bdcba16-fb70-4578-b415-23549724b425-kube-api-access-mshjr\") pod \"cert-manager-operator-controller-manager-55c444bf86-ddstf\" (UID: \"8bdcba16-fb70-4578-b415-23549724b425\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.363046 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.362995 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bdcba16-fb70-4578-b415-23549724b425-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ddstf\" (UID: \"8bdcba16-fb70-4578-b415-23549724b425\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.363372 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.363355 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bdcba16-fb70-4578-b415-23549724b425-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ddstf\" (UID: \"8bdcba16-fb70-4578-b415-23549724b425\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.372487 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.372460 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mshjr\" (UniqueName: \"kubernetes.io/projected/8bdcba16-fb70-4578-b415-23549724b425-kube-api-access-mshjr\") pod \"cert-manager-operator-controller-manager-55c444bf86-ddstf\" (UID: \"8bdcba16-fb70-4578-b415-23549724b425\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.503541 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.503509 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" Jul 18 11:25:31.645217 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:31.645192 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf"] Jul 18 11:25:31.648249 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:25:31.648223 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdcba16_fb70_4578_b415_23549724b425.slice/crio-fffd40df590516bbdbfb54a8fa544e9f3cdb48fa81acfbbc21d25cb09cd97cd7 WatchSource:0}: Error finding container fffd40df590516bbdbfb54a8fa544e9f3cdb48fa81acfbbc21d25cb09cd97cd7: Status 404 returned error can't find the container with id fffd40df590516bbdbfb54a8fa544e9f3cdb48fa81acfbbc21d25cb09cd97cd7 Jul 18 11:25:32.227083 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:32.227040 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" event={"ID":"8bdcba16-fb70-4578-b415-23549724b425","Type":"ContainerStarted","Data":"fffd40df590516bbdbfb54a8fa544e9f3cdb48fa81acfbbc21d25cb09cd97cd7"} Jul 18 11:25:32.884950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:32.884914 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:32.985437 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:32.985388 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:33.233496 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:33.233454 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" event={"ID":"8bdcba16-fb70-4578-b415-23549724b425","Type":"ContainerStarted","Data":"96a9533ca6cf0f22faad58b682f8adda372859bc247c9d64e8bbc842d31a17a5"} Jul 18 11:25:33.258808 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:33.258748 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ddstf" podStartSLOduration=0.802428248 podStartE2EDuration="2.258729961s" totalImagesPullingTime="1.456301713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:25:33.256527123 +0000 UTC m=+328.158291262" watchObservedRunningTime="2026-07-18 11:25:33.258729961 +0000 UTC m=+328.160494098" Jul 18 11:25:33.687245 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:33.687204 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:33.816398 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:33.816349 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:34.495049 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.495011 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh"] Jul 18 11:25:34.498558 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.498537 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.501410 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.501382 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 11:25:34.501563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.501382 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 11:25:34.501563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.501383 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9tnk2\"" Jul 18 11:25:34.507963 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.507929 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh"] Jul 18 11:25:34.587696 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.587651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcmc\" (UniqueName: \"kubernetes.io/projected/027db25b-1dce-47df-b992-4bea1420ef4a-kube-api-access-btcmc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.587973 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.587812 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.587973 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.587842 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.688846 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.688790 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.688846 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.688849 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.689239 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.688914 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcmc\" (UniqueName: \"kubernetes.io/projected/027db25b-1dce-47df-b992-4bea1420ef4a-kube-api-access-btcmc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.689239 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.689225 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.689336 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.689256 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.697726 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.697686 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcmc\" (UniqueName: \"kubernetes.io/projected/027db25b-1dce-47df-b992-4bea1420ef4a-kube-api-access-btcmc\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.809329 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.809289 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:34.903911 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.903836 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:34.968984 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:34.968950 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh"] Jul 18 11:25:34.970828 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:25:34.970802 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027db25b_1dce_47df_b992_4bea1420ef4a.slice/crio-ba593d036c0417984e6faacf8191c2294a6a84e5b1566084eba8517f494a79ca WatchSource:0}: Error finding container ba593d036c0417984e6faacf8191c2294a6a84e5b1566084eba8517f494a79ca: Status 404 returned error can't find the container with id ba593d036c0417984e6faacf8191c2294a6a84e5b1566084eba8517f494a79ca Jul 18 11:25:35.009893 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.009853 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:35.088045 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.088010 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8ppn7"] Jul 18 11:25:35.091266 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.091243 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.094203 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.094180 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 18 11:25:35.094323 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.094205 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 18 11:25:35.094323 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.094219 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9hh2n\"" Jul 18 11:25:35.100787 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.100760 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8ppn7"] Jul 18 11:25:35.194052 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.194008 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f10afee-e757-404a-be96-2bc3e064a23c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8ppn7\" (UID: \"0f10afee-e757-404a-be96-2bc3e064a23c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.194052 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.194051 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b54tl\" (UniqueName: \"kubernetes.io/projected/0f10afee-e757-404a-be96-2bc3e064a23c-kube-api-access-b54tl\") pod \"cert-manager-webhook-6486ffd997-8ppn7\" (UID: \"0f10afee-e757-404a-be96-2bc3e064a23c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.243359 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.243325 2585 generic.go:356] "Generic (PLEG): container finished" podID="027db25b-1dce-47df-b992-4bea1420ef4a" containerID="e97139ec0a144588f91cad24529fa9222b5b407d2203d005865b76126b02beb2" exitCode=0 Jul 18 11:25:35.243359 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.243363 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" event={"ID":"027db25b-1dce-47df-b992-4bea1420ef4a","Type":"ContainerDied","Data":"e97139ec0a144588f91cad24529fa9222b5b407d2203d005865b76126b02beb2"} Jul 18 11:25:35.243565 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.243397 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" event={"ID":"027db25b-1dce-47df-b992-4bea1420ef4a","Type":"ContainerStarted","Data":"ba593d036c0417984e6faacf8191c2294a6a84e5b1566084eba8517f494a79ca"} Jul 18 11:25:35.294602 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.294568 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f10afee-e757-404a-be96-2bc3e064a23c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8ppn7\" (UID: \"0f10afee-e757-404a-be96-2bc3e064a23c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.294602 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.294603 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b54tl\" (UniqueName: \"kubernetes.io/projected/0f10afee-e757-404a-be96-2bc3e064a23c-kube-api-access-b54tl\") pod \"cert-manager-webhook-6486ffd997-8ppn7\" (UID: \"0f10afee-e757-404a-be96-2bc3e064a23c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.303201 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.303156 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f10afee-e757-404a-be96-2bc3e064a23c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8ppn7\" (UID: \"0f10afee-e757-404a-be96-2bc3e064a23c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.303329 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.303257 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b54tl\" (UniqueName: \"kubernetes.io/projected/0f10afee-e757-404a-be96-2bc3e064a23c-kube-api-access-b54tl\") pod \"cert-manager-webhook-6486ffd997-8ppn7\" (UID: \"0f10afee-e757-404a-be96-2bc3e064a23c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.444941 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.444822 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:35.585879 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:35.585849 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8ppn7"] Jul 18 11:25:35.586756 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:25:35.586731 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f10afee_e757_404a_be96_2bc3e064a23c.slice/crio-ee1b95df9564fd41e6b91e3ec243ef1aa12d9c5f921042cb5ccaf5a6948d2b42 WatchSource:0}: Error finding container ee1b95df9564fd41e6b91e3ec243ef1aa12d9c5f921042cb5ccaf5a6948d2b42: Status 404 returned error can't find the container with id ee1b95df9564fd41e6b91e3ec243ef1aa12d9c5f921042cb5ccaf5a6948d2b42 Jul 18 11:25:36.251104 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:36.251065 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" event={"ID":"0f10afee-e757-404a-be96-2bc3e064a23c","Type":"ContainerStarted","Data":"ee1b95df9564fd41e6b91e3ec243ef1aa12d9c5f921042cb5ccaf5a6948d2b42"} Jul 18 11:25:37.293106 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:37.293070 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:37.555415 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:37.555317 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:38.266943 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:38.266861 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" event={"ID":"027db25b-1dce-47df-b992-4bea1420ef4a","Type":"ContainerStarted","Data":"a1af4f1fd2b49323ae8f1e1c7e454528ce28ab1cbdf1b02456235c9b873fd523"} Jul 18 11:25:39.273328 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:39.273293 2585 generic.go:356] "Generic (PLEG): container finished" podID="027db25b-1dce-47df-b992-4bea1420ef4a" containerID="a1af4f1fd2b49323ae8f1e1c7e454528ce28ab1cbdf1b02456235c9b873fd523" exitCode=0 Jul 18 11:25:39.273800 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:39.273369 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" event={"ID":"027db25b-1dce-47df-b992-4bea1420ef4a","Type":"ContainerDied","Data":"a1af4f1fd2b49323ae8f1e1c7e454528ce28ab1cbdf1b02456235c9b873fd523"} Jul 18 11:25:39.274968 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:39.274944 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" event={"ID":"0f10afee-e757-404a-be96-2bc3e064a23c","Type":"ContainerStarted","Data":"0ca6de214aa9c2e91d5891015bed2f2d6900cc0ce194ede7e89ad29d199743d1"} Jul 18 11:25:39.275096 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:39.275078 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:39.306164 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:39.306109 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" podStartSLOduration=1.5540769650000001 podStartE2EDuration="4.306089506s" totalImagesPullingTime="2.752012541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:25:39.304979948 +0000 UTC m=+334.206744090" watchObservedRunningTime="2026-07-18 11:25:39.306089506 +0000 UTC m=+334.207853643" Jul 18 11:25:39.680490 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:39.680386 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:39.772166 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:39.772129 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:40.280878 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:40.280841 2585 generic.go:356] "Generic (PLEG): container finished" podID="027db25b-1dce-47df-b992-4bea1420ef4a" containerID="a59d5c9f3ded85244ea2d7daac980045520403449272e51488feb069c7b09b4b" exitCode=0 Jul 18 11:25:40.281326 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:40.280914 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" event={"ID":"027db25b-1dce-47df-b992-4bea1420ef4a","Type":"ContainerDied","Data":"a59d5c9f3ded85244ea2d7daac980045520403449272e51488feb069c7b09b4b"} Jul 18 11:25:41.421501 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.421470 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:41.552275 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.552163 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-bundle\") pod \"027db25b-1dce-47df-b992-4bea1420ef4a\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " Jul 18 11:25:41.552275 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.552252 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-util\") pod \"027db25b-1dce-47df-b992-4bea1420ef4a\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " Jul 18 11:25:41.552476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.552316 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btcmc\" (UniqueName: \"kubernetes.io/projected/027db25b-1dce-47df-b992-4bea1420ef4a-kube-api-access-btcmc\") pod \"027db25b-1dce-47df-b992-4bea1420ef4a\" (UID: \"027db25b-1dce-47df-b992-4bea1420ef4a\") " Jul 18 11:25:41.552644 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.552622 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-bundle" (OuterVolumeSpecName: "bundle") pod "027db25b-1dce-47df-b992-4bea1420ef4a" (UID: "027db25b-1dce-47df-b992-4bea1420ef4a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:25:41.554460 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.554437 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027db25b-1dce-47df-b992-4bea1420ef4a-kube-api-access-btcmc" (OuterVolumeSpecName: "kube-api-access-btcmc") pod "027db25b-1dce-47df-b992-4bea1420ef4a" (UID: "027db25b-1dce-47df-b992-4bea1420ef4a"). InnerVolumeSpecName "kube-api-access-btcmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:25:41.556606 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.556558 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-util" (OuterVolumeSpecName: "util") pod "027db25b-1dce-47df-b992-4bea1420ef4a" (UID: "027db25b-1dce-47df-b992-4bea1420ef4a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:25:41.653253 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.653212 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btcmc\" (UniqueName: \"kubernetes.io/projected/027db25b-1dce-47df-b992-4bea1420ef4a-kube-api-access-btcmc\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:25:41.653253 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.653249 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:25:41.653253 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:41.653258 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/027db25b-1dce-47df-b992-4bea1420ef4a-util\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:25:42.291449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:42.291411 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" event={"ID":"027db25b-1dce-47df-b992-4bea1420ef4a","Type":"ContainerDied","Data":"ba593d036c0417984e6faacf8191c2294a6a84e5b1566084eba8517f494a79ca"} Jul 18 11:25:42.291449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:42.291431 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjdqbh" Jul 18 11:25:42.291655 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:42.291459 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba593d036c0417984e6faacf8191c2294a6a84e5b1566084eba8517f494a79ca" Jul 18 11:25:44.871239 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:44.871201 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:45.283375 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:45.283344 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8ppn7" Jul 18 11:25:48.282559 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282523 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w6ssc"] Jul 18 11:25:48.282955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282839 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027db25b-1dce-47df-b992-4bea1420ef4a" containerName="extract" Jul 18 11:25:48.282955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282851 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="027db25b-1dce-47df-b992-4bea1420ef4a" containerName="extract" Jul 18 11:25:48.282955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282864 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027db25b-1dce-47df-b992-4bea1420ef4a" containerName="util" Jul 18 11:25:48.282955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282869 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="027db25b-1dce-47df-b992-4bea1420ef4a" containerName="util" Jul 18 11:25:48.282955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282895 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027db25b-1dce-47df-b992-4bea1420ef4a" containerName="pull" Jul 18 11:25:48.282955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282904 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="027db25b-1dce-47df-b992-4bea1420ef4a" containerName="pull" Jul 18 11:25:48.283141 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.282990 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="027db25b-1dce-47df-b992-4bea1420ef4a" containerName="extract" Jul 18 11:25:48.286605 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.286586 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.289078 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.289056 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ml77l\"" Jul 18 11:25:48.293463 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.293104 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w6ssc"] Jul 18 11:25:48.412622 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.412588 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcp6b\" (UniqueName: \"kubernetes.io/projected/889e961f-1392-4340-bc4c-5430bd51353c-kube-api-access-vcp6b\") pod \"cert-manager-55b4d954d5-w6ssc\" (UID: \"889e961f-1392-4340-bc4c-5430bd51353c\") " pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.412796 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.412627 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/889e961f-1392-4340-bc4c-5430bd51353c-bound-sa-token\") pod \"cert-manager-55b4d954d5-w6ssc\" (UID: \"889e961f-1392-4340-bc4c-5430bd51353c\") " pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.513198 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.513142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcp6b\" (UniqueName: \"kubernetes.io/projected/889e961f-1392-4340-bc4c-5430bd51353c-kube-api-access-vcp6b\") pod \"cert-manager-55b4d954d5-w6ssc\" (UID: \"889e961f-1392-4340-bc4c-5430bd51353c\") " pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.513198 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.513201 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/889e961f-1392-4340-bc4c-5430bd51353c-bound-sa-token\") pod \"cert-manager-55b4d954d5-w6ssc\" (UID: \"889e961f-1392-4340-bc4c-5430bd51353c\") " pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.522657 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.522623 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcp6b\" (UniqueName: \"kubernetes.io/projected/889e961f-1392-4340-bc4c-5430bd51353c-kube-api-access-vcp6b\") pod \"cert-manager-55b4d954d5-w6ssc\" (UID: \"889e961f-1392-4340-bc4c-5430bd51353c\") " pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.522657 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.522624 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/889e961f-1392-4340-bc4c-5430bd51353c-bound-sa-token\") pod \"cert-manager-55b4d954d5-w6ssc\" (UID: \"889e961f-1392-4340-bc4c-5430bd51353c\") " pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.595938 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.595822 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w6ssc" Jul 18 11:25:48.730762 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:48.730726 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w6ssc"] Jul 18 11:25:48.734213 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:25:48.734181 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod889e961f_1392_4340_bc4c_5430bd51353c.slice/crio-ae349ceec65da207a4c1c88cfc4a7adb71e2df4d77e887e6c5052b0297bfe13b WatchSource:0}: Error finding container ae349ceec65da207a4c1c88cfc4a7adb71e2df4d77e887e6c5052b0297bfe13b: Status 404 returned error can't find the container with id ae349ceec65da207a4c1c88cfc4a7adb71e2df4d77e887e6c5052b0297bfe13b Jul 18 11:25:49.322052 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:49.322015 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w6ssc" event={"ID":"889e961f-1392-4340-bc4c-5430bd51353c","Type":"ContainerStarted","Data":"2afb8d6e6c925b79aacfa2b294ff6651a4ba3e9edd5cf209adf2ac55127b73fc"} Jul 18 11:25:49.322490 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:49.322062 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w6ssc" event={"ID":"889e961f-1392-4340-bc4c-5430bd51353c","Type":"ContainerStarted","Data":"ae349ceec65da207a4c1c88cfc4a7adb71e2df4d77e887e6c5052b0297bfe13b"} Jul 18 11:25:49.338969 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:49.338923 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-w6ssc" podStartSLOduration=1.3389041320000001 podStartE2EDuration="1.338904132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:25:49.338559595 +0000 UTC m=+344.240323734" watchObservedRunningTime="2026-07-18 11:25:49.338904132 +0000 UTC m=+344.240668261" Jul 18 11:25:50.290960 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:50.290918 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:50.386741 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:50.386703 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:52.303146 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.303107 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp"] Jul 18 11:25:52.306745 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.306727 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.309483 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.309459 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 18 11:25:52.309595 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.309459 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:25:52.309595 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.309567 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lk2pf\"" Jul 18 11:25:52.316166 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.316138 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp"] Jul 18 11:25:52.340677 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.340641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98fa54ac-f9ae-40a0-aae7-617d6178e5fa-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tv2kp\" (UID: \"98fa54ac-f9ae-40a0-aae7-617d6178e5fa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.340865 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.340716 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9r2\" (UniqueName: \"kubernetes.io/projected/98fa54ac-f9ae-40a0-aae7-617d6178e5fa-kube-api-access-nw9r2\") pod \"openshift-lws-operator-fd8ccff4c-tv2kp\" (UID: \"98fa54ac-f9ae-40a0-aae7-617d6178e5fa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.441542 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.441492 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9r2\" (UniqueName: \"kubernetes.io/projected/98fa54ac-f9ae-40a0-aae7-617d6178e5fa-kube-api-access-nw9r2\") pod \"openshift-lws-operator-fd8ccff4c-tv2kp\" (UID: \"98fa54ac-f9ae-40a0-aae7-617d6178e5fa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.441542 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.441555 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98fa54ac-f9ae-40a0-aae7-617d6178e5fa-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tv2kp\" (UID: \"98fa54ac-f9ae-40a0-aae7-617d6178e5fa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.441982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.441963 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98fa54ac-f9ae-40a0-aae7-617d6178e5fa-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tv2kp\" (UID: \"98fa54ac-f9ae-40a0-aae7-617d6178e5fa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.450930 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.450904 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9r2\" (UniqueName: \"kubernetes.io/projected/98fa54ac-f9ae-40a0-aae7-617d6178e5fa-kube-api-access-nw9r2\") pod \"openshift-lws-operator-fd8ccff4c-tv2kp\" (UID: \"98fa54ac-f9ae-40a0-aae7-617d6178e5fa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.617150 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.617060 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" Jul 18 11:25:52.753140 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:52.753108 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp"] Jul 18 11:25:52.755204 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:25:52.755151 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98fa54ac_f9ae_40a0_aae7_617d6178e5fa.slice/crio-56fe321633497c828d139b6b82abfca87864c19a0c028254166c28f7341332a3 WatchSource:0}: Error finding container 56fe321633497c828d139b6b82abfca87864c19a0c028254166c28f7341332a3: Status 404 returned error can't find the container with id 56fe321633497c828d139b6b82abfca87864c19a0c028254166c28f7341332a3 Jul 18 11:25:53.343676 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:53.343642 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" event={"ID":"98fa54ac-f9ae-40a0-aae7-617d6178e5fa","Type":"ContainerStarted","Data":"56fe321633497c828d139b6b82abfca87864c19a0c028254166c28f7341332a3"} Jul 18 11:25:53.669148 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:53.669018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:55.354560 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:55.354524 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" event={"ID":"98fa54ac-f9ae-40a0-aae7-617d6178e5fa","Type":"ContainerStarted","Data":"dd1e2d596668b6270e6b455bff7f7d91a067327e9b10e3e31ae934095b4cdd37"} Jul 18 11:25:55.373396 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:55.373342 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv2kp" podStartSLOduration=1.833297343 podStartE2EDuration="3.373322343s" totalImagesPullingTime="1.540025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:25:55.372260889 +0000 UTC m=+350.274025027" watchObservedRunningTime="2026-07-18 11:25:55.373322343 +0000 UTC m=+350.275086484" Jul 18 11:25:56.058843 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:56.058797 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:56.157017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:56.156975 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:25:57.897227 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:25:57.897126 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:08.600850 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:08.600810 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:08.649214 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:08.649177 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:09.959448 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:09.959409 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29"] Jul 18 11:26:09.965569 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:09.965544 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:09.968195 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:09.968172 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 11:26:09.969279 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:09.969261 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9tnk2\"" Jul 18 11:26:09.969279 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:09.969271 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 11:26:09.973245 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:09.973178 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29"] Jul 18 11:26:10.093440 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.093403 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.093621 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.093489 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.093621 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.093518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpjp7\" (UniqueName: \"kubernetes.io/projected/b53b0775-6231-4118-8fe0-05a0782d3825-kube-api-access-vpjp7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.194424 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.194382 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.194630 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.194474 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.194630 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.194507 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpjp7\" (UniqueName: \"kubernetes.io/projected/b53b0775-6231-4118-8fe0-05a0782d3825-kube-api-access-vpjp7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.194943 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.194917 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.195022 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.194916 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.203775 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.203745 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpjp7\" (UniqueName: \"kubernetes.io/projected/b53b0775-6231-4118-8fe0-05a0782d3825-kube-api-access-vpjp7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.276768 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.276730 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:10.446422 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:10.446386 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29"] Jul 18 11:26:10.449071 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:26:10.449044 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53b0775_6231_4118_8fe0_05a0782d3825.slice/crio-184193eea7bd97d5002690c23a8815b3b98d92739502356f87678e4b36a95142 WatchSource:0}: Error finding container 184193eea7bd97d5002690c23a8815b3b98d92739502356f87678e4b36a95142: Status 404 returned error can't find the container with id 184193eea7bd97d5002690c23a8815b3b98d92739502356f87678e4b36a95142 Jul 18 11:26:11.431925 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:11.431867 2585 generic.go:356] "Generic (PLEG): container finished" podID="b53b0775-6231-4118-8fe0-05a0782d3825" containerID="b37917ea0304d21a27587436f9f203d0710055282cd70584b17a4c46051189bb" exitCode=0 Jul 18 11:26:11.432402 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:11.431921 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" event={"ID":"b53b0775-6231-4118-8fe0-05a0782d3825","Type":"ContainerDied","Data":"b37917ea0304d21a27587436f9f203d0710055282cd70584b17a4c46051189bb"} Jul 18 11:26:11.432402 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:11.431996 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" event={"ID":"b53b0775-6231-4118-8fe0-05a0782d3825","Type":"ContainerStarted","Data":"184193eea7bd97d5002690c23a8815b3b98d92739502356f87678e4b36a95142"} Jul 18 11:26:12.438506 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:12.438420 2585 generic.go:356] "Generic (PLEG): container finished" podID="b53b0775-6231-4118-8fe0-05a0782d3825" containerID="90d03353a79d23ae960484fb9a34119be49ef4c7b189cc11381be7780874df5d" exitCode=0 Jul 18 11:26:12.438506 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:12.438485 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" event={"ID":"b53b0775-6231-4118-8fe0-05a0782d3825","Type":"ContainerDied","Data":"90d03353a79d23ae960484fb9a34119be49ef4c7b189cc11381be7780874df5d"} Jul 18 11:26:13.444802 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:13.444765 2585 generic.go:356] "Generic (PLEG): container finished" podID="b53b0775-6231-4118-8fe0-05a0782d3825" containerID="c4b1d067c9c6815291affebbca93eddc1cf4c22f00055e29b8cbe6f28fbd6b4b" exitCode=0 Jul 18 11:26:13.445231 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:13.444839 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" event={"ID":"b53b0775-6231-4118-8fe0-05a0782d3825","Type":"ContainerDied","Data":"c4b1d067c9c6815291affebbca93eddc1cf4c22f00055e29b8cbe6f28fbd6b4b"} Jul 18 11:26:14.136266 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.136226 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:14.240229 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.240193 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw"] Jul 18 11:26:14.243559 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.243533 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.246821 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.246798 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 18 11:26:14.251584 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.251563 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 18 11:26:14.251704 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.251610 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b96ln\"" Jul 18 11:26:14.254330 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.254307 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 18 11:26:14.264591 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.264557 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:14.264982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.264961 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw"] Jul 18 11:26:14.331006 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.330962 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4b9fa335-2643-477c-a8c2-155e4da1bdf4-manager-config\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.331006 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.331008 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b9fa335-2643-477c-a8c2-155e4da1bdf4-cert\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.331289 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.331066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b9fa335-2643-477c-a8c2-155e4da1bdf4-metrics-cert\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.331289 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.331098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5tkg\" (UniqueName: \"kubernetes.io/projected/4b9fa335-2643-477c-a8c2-155e4da1bdf4-kube-api-access-w5tkg\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.432509 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.432416 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4b9fa335-2643-477c-a8c2-155e4da1bdf4-manager-config\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.432509 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.432466 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b9fa335-2643-477c-a8c2-155e4da1bdf4-cert\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.432509 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.432494 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b9fa335-2643-477c-a8c2-155e4da1bdf4-metrics-cert\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.432748 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.432512 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5tkg\" (UniqueName: \"kubernetes.io/projected/4b9fa335-2643-477c-a8c2-155e4da1bdf4-kube-api-access-w5tkg\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.433179 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.433146 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4b9fa335-2643-477c-a8c2-155e4da1bdf4-manager-config\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.435103 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.435081 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b9fa335-2643-477c-a8c2-155e4da1bdf4-cert\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.435247 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.435228 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b9fa335-2643-477c-a8c2-155e4da1bdf4-metrics-cert\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.443385 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.443353 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5tkg\" (UniqueName: \"kubernetes.io/projected/4b9fa335-2643-477c-a8c2-155e4da1bdf4-kube-api-access-w5tkg\") pod \"lws-controller-manager-59bdd75cb8-f4lkw\" (UID: \"4b9fa335-2643-477c-a8c2-155e4da1bdf4\") " pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.555139 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.555097 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:14.588640 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.588610 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:14.697934 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.697682 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw"] Jul 18 11:26:14.700695 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:26:14.700666 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9fa335_2643_477c_a8c2_155e4da1bdf4.slice/crio-9cdc41e7400d9a79ea42ed8c0fc9c1850a33fe278c56d91c784af0269e1166cc WatchSource:0}: Error finding container 9cdc41e7400d9a79ea42ed8c0fc9c1850a33fe278c56d91c784af0269e1166cc: Status 404 returned error can't find the container with id 9cdc41e7400d9a79ea42ed8c0fc9c1850a33fe278c56d91c784af0269e1166cc Jul 18 11:26:14.735117 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.735081 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-bundle\") pod \"b53b0775-6231-4118-8fe0-05a0782d3825\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " Jul 18 11:26:14.735333 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.735142 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpjp7\" (UniqueName: \"kubernetes.io/projected/b53b0775-6231-4118-8fe0-05a0782d3825-kube-api-access-vpjp7\") pod \"b53b0775-6231-4118-8fe0-05a0782d3825\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " Jul 18 11:26:14.735333 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.735234 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-util\") pod \"b53b0775-6231-4118-8fe0-05a0782d3825\" (UID: \"b53b0775-6231-4118-8fe0-05a0782d3825\") " Jul 18 11:26:14.735928 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.735863 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-bundle" (OuterVolumeSpecName: "bundle") pod "b53b0775-6231-4118-8fe0-05a0782d3825" (UID: "b53b0775-6231-4118-8fe0-05a0782d3825"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:26:14.737369 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.737339 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b53b0775-6231-4118-8fe0-05a0782d3825-kube-api-access-vpjp7" (OuterVolumeSpecName: "kube-api-access-vpjp7") pod "b53b0775-6231-4118-8fe0-05a0782d3825" (UID: "b53b0775-6231-4118-8fe0-05a0782d3825"). InnerVolumeSpecName "kube-api-access-vpjp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:26:14.740826 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.740772 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-util" (OuterVolumeSpecName: "util") pod "b53b0775-6231-4118-8fe0-05a0782d3825" (UID: "b53b0775-6231-4118-8fe0-05a0782d3825"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:26:14.835848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.835806 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:26:14.835848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.835844 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpjp7\" (UniqueName: \"kubernetes.io/projected/b53b0775-6231-4118-8fe0-05a0782d3825-kube-api-access-vpjp7\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:26:14.835848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:14.835857 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b0775-6231-4118-8fe0-05a0782d3825-util\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:26:15.456477 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:15.456437 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" event={"ID":"4b9fa335-2643-477c-a8c2-155e4da1bdf4","Type":"ContainerStarted","Data":"9cdc41e7400d9a79ea42ed8c0fc9c1850a33fe278c56d91c784af0269e1166cc"} Jul 18 11:26:15.458567 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:15.458541 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" Jul 18 11:26:15.458718 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:15.458537 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4fj29" event={"ID":"b53b0775-6231-4118-8fe0-05a0782d3825","Type":"ContainerDied","Data":"184193eea7bd97d5002690c23a8815b3b98d92739502356f87678e4b36a95142"} Jul 18 11:26:15.458718 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:15.458681 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="184193eea7bd97d5002690c23a8815b3b98d92739502356f87678e4b36a95142" Jul 18 11:26:16.465724 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:16.465625 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" event={"ID":"4b9fa335-2643-477c-a8c2-155e4da1bdf4","Type":"ContainerStarted","Data":"6d1d135e328e2db93d6523eee4a0573ee84b465d74843611ea94d0855128a5a5"} Jul 18 11:26:16.465724 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:16.465719 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:16.482984 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:16.482927 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" podStartSLOduration=1.024565927 podStartE2EDuration="2.482909965s" totalImagesPullingTime="1.458344038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:26:16.481442024 +0000 UTC m=+371.383206152" watchObservedRunningTime="2026-07-18 11:26:16.482909965 +0000 UTC m=+371.384674098" Jul 18 11:26:21.975352 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:21.975314 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:22.074370 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.074334 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:22.378018 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.377981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:22.572648 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.572618 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:22.673737 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.673650 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:22.706148 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706111 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657"] Jul 18 11:26:22.706553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706528 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b53b0775-6231-4118-8fe0-05a0782d3825" containerName="extract" Jul 18 11:26:22.706553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706550 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b53b0775-6231-4118-8fe0-05a0782d3825" containerName="extract" Jul 18 11:26:22.706751 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706583 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b53b0775-6231-4118-8fe0-05a0782d3825" containerName="pull" Jul 18 11:26:22.706751 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706592 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b53b0775-6231-4118-8fe0-05a0782d3825" containerName="pull" Jul 18 11:26:22.706751 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706601 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b53b0775-6231-4118-8fe0-05a0782d3825" containerName="util" Jul 18 11:26:22.706751 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706610 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b53b0775-6231-4118-8fe0-05a0782d3825" containerName="util" Jul 18 11:26:22.706751 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.706699 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b53b0775-6231-4118-8fe0-05a0782d3825" containerName="extract" Jul 18 11:26:22.710136 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.710115 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.713082 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.713038 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 11:26:22.713260 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.713090 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 11:26:22.713260 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.713095 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 11:26:22.713420 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.713376 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-58g65\"" Jul 18 11:26:22.713420 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.713408 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 11:26:22.730540 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.730507 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657"] Jul 18 11:26:22.810184 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.810146 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx24k\" (UniqueName: \"kubernetes.io/projected/d173da93-9010-42f8-9d6b-056af8fe7613-kube-api-access-zx24k\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.810403 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.810279 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d173da93-9010-42f8-9d6b-056af8fe7613-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.810403 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.810323 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d173da93-9010-42f8-9d6b-056af8fe7613-tmp\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.810403 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.810347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d173da93-9010-42f8-9d6b-056af8fe7613-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.810403 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.810383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d173da93-9010-42f8-9d6b-056af8fe7613-manifests\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.865811 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.865773 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:22.911778 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.911730 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d173da93-9010-42f8-9d6b-056af8fe7613-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.912004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.911785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d173da93-9010-42f8-9d6b-056af8fe7613-tmp\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.912004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.911929 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d173da93-9010-42f8-9d6b-056af8fe7613-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.912004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.911977 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d173da93-9010-42f8-9d6b-056af8fe7613-manifests\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.912004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.912005 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx24k\" (UniqueName: \"kubernetes.io/projected/d173da93-9010-42f8-9d6b-056af8fe7613-kube-api-access-zx24k\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.912389 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.912360 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d173da93-9010-42f8-9d6b-056af8fe7613-manifests\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.914297 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.914273 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d173da93-9010-42f8-9d6b-056af8fe7613-tmp\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.914375 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.914359 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d173da93-9010-42f8-9d6b-056af8fe7613-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.914463 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.914448 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d173da93-9010-42f8-9d6b-056af8fe7613-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.920167 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.920140 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx24k\" (UniqueName: \"kubernetes.io/projected/d173da93-9010-42f8-9d6b-056af8fe7613-kube-api-access-zx24k\") pod \"opendatahub-operator-controller-manager-78b8cdd6f7-gn657\" (UID: \"d173da93-9010-42f8-9d6b-056af8fe7613\") " pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:22.962526 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:22.962433 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:23.020451 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.020403 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:23.063450 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.063412 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:23.136468 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.136425 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:23.167229 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.167203 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657"] Jul 18 11:26:23.170021 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:26:23.169990 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd173da93_9010_42f8_9d6b_056af8fe7613.slice/crio-96796b4f759d6bee5be42530fe1d28d94c907c0653f4e9854a63295ac15d1948 WatchSource:0}: Error finding container 96796b4f759d6bee5be42530fe1d28d94c907c0653f4e9854a63295ac15d1948: Status 404 returned error can't find the container with id 96796b4f759d6bee5be42530fe1d28d94c907c0653f4e9854a63295ac15d1948 Jul 18 11:26:23.195134 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.195101 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:23.296005 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.295957 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:23.398022 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.397987 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:23.502547 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:23.502509 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" event={"ID":"d173da93-9010-42f8-9d6b-056af8fe7613","Type":"ContainerStarted","Data":"96796b4f759d6bee5be42530fe1d28d94c907c0653f4e9854a63295ac15d1948"} Jul 18 11:26:26.519555 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:26.519512 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" event={"ID":"d173da93-9010-42f8-9d6b-056af8fe7613","Type":"ContainerStarted","Data":"efd19a1fd17efb7d9b002445bddf4d669796117761c5617ef139d4aff8689774"} Jul 18 11:26:27.473568 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:27.473539 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59bdd75cb8-f4lkw" Jul 18 11:26:27.526075 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:27.526034 2585 generic.go:356] "Generic (PLEG): container finished" podID="d173da93-9010-42f8-9d6b-056af8fe7613" containerID="efd19a1fd17efb7d9b002445bddf4d669796117761c5617ef139d4aff8689774" exitCode=0 Jul 18 11:26:27.526547 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:27.526123 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" event={"ID":"d173da93-9010-42f8-9d6b-056af8fe7613","Type":"ContainerDied","Data":"efd19a1fd17efb7d9b002445bddf4d669796117761c5617ef139d4aff8689774"} Jul 18 11:26:28.532743 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:28.532710 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" event={"ID":"d173da93-9010-42f8-9d6b-056af8fe7613","Type":"ContainerStarted","Data":"ed5132e5fd05403c36e3b04097fabe0c1166eb25866770002d40d2b606f42985"} Jul 18 11:26:28.533180 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:28.532842 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:28.555599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:28.555541 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" podStartSLOduration=2.190666527 podStartE2EDuration="6.555522563s" totalImagesPullingTime="3.320822185s" totalInitContainerRuntime="1.044033851s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:26:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:26:28.553693742 +0000 UTC m=+383.455457918" watchObservedRunningTime="2026-07-18 11:26:28.555522563 +0000 UTC m=+383.457286700" Jul 18 11:26:30.280505 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:30.280469 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:39.539295 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:39.539264 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78b8cdd6f7-gn657" Jul 18 11:26:42.036606 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:42.036564 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:42.159820 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:42.159773 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:42.750213 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:42.750175 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:42.937302 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:42.937267 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:43.000558 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:43.000472 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:43.100756 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:43.100707 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:43.203834 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:43.203801 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:43.300568 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:43.300534 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:43.775773 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:43.775739 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:43.953542 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:43.953501 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:44.273262 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:44.273229 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:44.338849 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:44.338812 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:45.370085 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:45.370050 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:49.963197 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:49.963143 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:50.079918 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.079865 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:50.160198 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.160163 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd"] Jul 18 11:26:50.171421 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.171388 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.174318 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.174280 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 18 11:26:50.175145 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.175088 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2q4cv\"" Jul 18 11:26:50.175682 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.175506 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 18 11:26:50.175682 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.175534 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 11:26:50.176255 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.176233 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 18 11:26:50.176371 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.176348 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 11:26:50.176446 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.176301 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 11:26:50.176765 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.176737 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd"] Jul 18 11:26:50.249740 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.249633 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.249740 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.249680 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.250017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.249777 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.250017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.249823 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.250017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.249862 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.250017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.249935 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.250017 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.249972 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r2mk\" (UniqueName: \"kubernetes.io/projected/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-kube-api-access-6r2mk\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.350992 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.350955 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.351215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.351013 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.351215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.351037 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.351215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.351071 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.351215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.351115 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r2mk\" (UniqueName: \"kubernetes.io/projected/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-kube-api-access-6r2mk\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.351215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.351146 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.351215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.351171 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.351944 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.351916 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.353512 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.353483 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.353637 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.353533 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.353637 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.353576 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.353725 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.353636 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.360181 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.360152 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.360306 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.360156 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r2mk\" (UniqueName: \"kubernetes.io/projected/bd5801a2-c674-4f9f-9122-74ec9f6eb6b0-kube-api-access-6r2mk\") pod \"istiod-openshift-gateway-94bb8fbfd-7t8qd\" (UID: \"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.483499 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.483447 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:50.659074 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.659022 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd"] Jul 18 11:26:50.660647 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:26:50.660604 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd5801a2_c674_4f9f_9122_74ec9f6eb6b0.slice/crio-249bfc4de4e58fdbe752fad7aac1e0260ba52ae2254bfea22a0007a6c6aba68e WatchSource:0}: Error finding container 249bfc4de4e58fdbe752fad7aac1e0260ba52ae2254bfea22a0007a6c6aba68e: Status 404 returned error can't find the container with id 249bfc4de4e58fdbe752fad7aac1e0260ba52ae2254bfea22a0007a6c6aba68e Jul 18 11:26:50.782345 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.782295 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:50.988441 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:50.988347 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:51.125047 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:51.125018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:51.316258 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:51.316161 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:51.413330 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:51.413295 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:51.648906 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:51.648789 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" event={"ID":"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0","Type":"ContainerStarted","Data":"249bfc4de4e58fdbe752fad7aac1e0260ba52ae2254bfea22a0007a6c6aba68e"} Jul 18 11:26:52.744599 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.744531 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:52.804419 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.804374 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj"] Jul 18 11:26:52.812000 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.811942 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:52.814948 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.814917 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-f5gp8\"" Jul 18 11:26:52.814948 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.814941 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 18 11:26:52.828692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.828160 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj"] Jul 18 11:26:52.842271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.842226 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:52.876808 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.876765 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04ef4102-2a02-4880-bcf7-0e8f31186a93-cert\") pod \"kserve-controller-manager-d6f4b65b8-kjlvj\" (UID: \"04ef4102-2a02-4880-bcf7-0e8f31186a93\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:52.877016 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.876935 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm7xb\" (UniqueName: \"kubernetes.io/projected/04ef4102-2a02-4880-bcf7-0e8f31186a93-kube-api-access-cm7xb\") pod \"kserve-controller-manager-d6f4b65b8-kjlvj\" (UID: \"04ef4102-2a02-4880-bcf7-0e8f31186a93\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:52.907348 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.907295 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:52.977750 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.977710 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm7xb\" (UniqueName: \"kubernetes.io/projected/04ef4102-2a02-4880-bcf7-0e8f31186a93-kube-api-access-cm7xb\") pod \"kserve-controller-manager-d6f4b65b8-kjlvj\" (UID: \"04ef4102-2a02-4880-bcf7-0e8f31186a93\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:52.977968 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.977811 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04ef4102-2a02-4880-bcf7-0e8f31186a93-cert\") pod \"kserve-controller-manager-d6f4b65b8-kjlvj\" (UID: \"04ef4102-2a02-4880-bcf7-0e8f31186a93\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:52.981096 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.981068 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04ef4102-2a02-4880-bcf7-0e8f31186a93-cert\") pod \"kserve-controller-manager-d6f4b65b8-kjlvj\" (UID: \"04ef4102-2a02-4880-bcf7-0e8f31186a93\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:52.997261 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:52.997181 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm7xb\" (UniqueName: \"kubernetes.io/projected/04ef4102-2a02-4880-bcf7-0e8f31186a93-kube-api-access-cm7xb\") pod \"kserve-controller-manager-d6f4b65b8-kjlvj\" (UID: \"04ef4102-2a02-4880-bcf7-0e8f31186a93\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:53.020644 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.020597 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:53.129774 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.129736 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 11:26:53.129964 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.129805 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 11:26:53.129964 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.129896 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:53.163872 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.163831 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:53.300445 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.300267 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj"] Jul 18 11:26:53.663063 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.662977 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" event={"ID":"bd5801a2-c674-4f9f-9122-74ec9f6eb6b0","Type":"ContainerStarted","Data":"cdbce6b30962bbbc648055cb49d0052a8cf75997662641c70c2c41237b212799"} Jul 18 11:26:53.663231 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.663078 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:53.664324 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.664295 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" event={"ID":"04ef4102-2a02-4880-bcf7-0e8f31186a93","Type":"ContainerStarted","Data":"8d4935ef03998b792968b477c1a716856f1653be297956bfff913930d05aef36"} Jul 18 11:26:53.725140 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:53.725083 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" podStartSLOduration=1.260414052 podStartE2EDuration="3.725061684s" totalImagesPullingTime="2.464647632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:26:53.725041122 +0000 UTC m=+408.626805263" watchObservedRunningTime="2026-07-18 11:26:53.725061684 +0000 UTC m=+408.626825814" Jul 18 11:26:54.029146 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:54.029085 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:54.128615 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:54.128569 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:26:54.672250 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:54.672220 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7t8qd" Jul 18 11:26:55.675438 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:55.675343 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" event={"ID":"04ef4102-2a02-4880-bcf7-0e8f31186a93","Type":"ContainerStarted","Data":"6f9451d1b91c9f8edc14b4e034ce759356dc7e1258a3c70f7ff42852ca3ba9d3"} Jul 18 11:26:55.675812 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:55.675491 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:26:55.695666 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:26:55.695609 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" podStartSLOduration=1.5822824949999998 podStartE2EDuration="3.695590646s" totalImagesPullingTime="2.113308151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:26:55.693564015 +0000 UTC m=+410.595328156" watchObservedRunningTime="2026-07-18 11:26:55.695590646 +0000 UTC m=+410.597354846" Jul 18 11:27:26.686137 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:26.686101 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kjlvj" Jul 18 11:27:40.328271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:40.328172 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:40.394063 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:40.394025 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:43.482102 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:43.482059 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:43.568645 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:43.568603 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:43.661386 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:43.661344 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:44.405170 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:44.405139 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:44.612732 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:44.612700 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:50.828828 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:50.828791 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b766c647-2snv8"] Jul 18 11:27:50.919375 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:50.919332 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:51.020483 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:51.020452 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:51.068080 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:51.068050 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:27:59.792502 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.792466 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd"] Jul 18 11:27:59.796072 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.796051 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.798662 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.798619 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 11:27:59.798818 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.798670 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 11:27:59.798818 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.798707 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-799hc\"" Jul 18 11:27:59.803315 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.803270 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd"] Jul 18 11:27:59.870315 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.870272 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-948rq\" (UniqueName: \"kubernetes.io/projected/8eee5fd2-40ff-4d9e-8701-83005930c682-kube-api-access-948rq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.870498 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.870330 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.870498 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.870358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.971247 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.971183 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.971247 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.971263 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.971466 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.971346 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-948rq\" (UniqueName: \"kubernetes.io/projected/8eee5fd2-40ff-4d9e-8701-83005930c682-kube-api-access-948rq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.971594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.971574 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.971656 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.971635 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:27:59.980094 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:27:59.980068 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-948rq\" (UniqueName: \"kubernetes.io/projected/8eee5fd2-40ff-4d9e-8701-83005930c682-kube-api-access-948rq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:28:00.106369 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.106271 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:28:00.256729 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.256697 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd"] Jul 18 11:28:00.258937 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:28:00.258910 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eee5fd2_40ff_4d9e_8701_83005930c682.slice/crio-93e216384c884214c1975b0a73f4e6910af13b0e7e39001df91caf0399162ce6 WatchSource:0}: Error finding container 93e216384c884214c1975b0a73f4e6910af13b0e7e39001df91caf0399162ce6: Status 404 returned error can't find the container with id 93e216384c884214c1975b0a73f4e6910af13b0e7e39001df91caf0399162ce6 Jul 18 11:28:00.595813 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.595768 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb"] Jul 18 11:28:00.601582 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.601529 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.604634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.604602 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb"] Jul 18 11:28:00.677014 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.676973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.677224 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.677025 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvjjg\" (UniqueName: \"kubernetes.io/projected/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-kube-api-access-vvjjg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.677224 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.677188 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.778347 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.778301 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.778347 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.778354 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.778600 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.778385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvjjg\" (UniqueName: \"kubernetes.io/projected/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-kube-api-access-vvjjg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.778727 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.778691 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.778798 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.778755 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.788417 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.788380 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvjjg\" (UniqueName: \"kubernetes.io/projected/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-kube-api-access-vvjjg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.911955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.911817 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:00.996600 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.996566 2585 generic.go:356] "Generic (PLEG): container finished" podID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerID="f3fab93f804740c73f84c8639e5e0d748a6817887140ab165977e4c4c4500c68" exitCode=0 Jul 18 11:28:00.996773 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.996662 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" event={"ID":"8eee5fd2-40ff-4d9e-8701-83005930c682","Type":"ContainerDied","Data":"f3fab93f804740c73f84c8639e5e0d748a6817887140ab165977e4c4c4500c68"} Jul 18 11:28:00.996773 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:00.996723 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" event={"ID":"8eee5fd2-40ff-4d9e-8701-83005930c682","Type":"ContainerStarted","Data":"93e216384c884214c1975b0a73f4e6910af13b0e7e39001df91caf0399162ce6"} Jul 18 11:28:01.053702 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.053675 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb"] Jul 18 11:28:01.055272 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:28:01.055246 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e097f8_f1b8_4c1d_a86a_0728eea9b93d.slice/crio-98117cd8b5178517739c62ad61b28caf376012ac8a70f235cfbbd6f941133308 WatchSource:0}: Error finding container 98117cd8b5178517739c62ad61b28caf376012ac8a70f235cfbbd6f941133308: Status 404 returned error can't find the container with id 98117cd8b5178517739c62ad61b28caf376012ac8a70f235cfbbd6f941133308 Jul 18 11:28:01.399027 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.398991 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz"] Jul 18 11:28:01.402689 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.402667 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.412462 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.412433 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz"] Jul 18 11:28:01.485758 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.485666 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.485758 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.485701 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.485758 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.485732 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsxjd\" (UniqueName: \"kubernetes.io/projected/c5e28c4d-981c-479c-9dc2-3a84f8769386-kube-api-access-rsxjd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.586526 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.586487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.586526 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.586526 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.586754 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.586574 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsxjd\" (UniqueName: \"kubernetes.io/projected/c5e28c4d-981c-479c-9dc2-3a84f8769386-kube-api-access-rsxjd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.586955 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.586922 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.587083 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.586978 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.595698 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.595663 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsxjd\" (UniqueName: \"kubernetes.io/projected/c5e28c4d-981c-479c-9dc2-3a84f8769386-kube-api-access-rsxjd\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.713244 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.713209 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:01.883503 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:01.883470 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz"] Jul 18 11:28:01.885266 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:28:01.885239 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e28c4d_981c_479c_9dc2_3a84f8769386.slice/crio-83d365771915aafebc07beacdbc28c5994d85a23ebda3e3ddabf56a16b5207ea WatchSource:0}: Error finding container 83d365771915aafebc07beacdbc28c5994d85a23ebda3e3ddabf56a16b5207ea: Status 404 returned error can't find the container with id 83d365771915aafebc07beacdbc28c5994d85a23ebda3e3ddabf56a16b5207ea Jul 18 11:28:02.003718 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.003689 2585 generic.go:356] "Generic (PLEG): container finished" podID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerID="3e02436f80a7e9b7cfbc51372083920e6a9ab133b219193d90f5c371a3957589" exitCode=0 Jul 18 11:28:02.004169 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.003775 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" event={"ID":"8eee5fd2-40ff-4d9e-8701-83005930c682","Type":"ContainerDied","Data":"3e02436f80a7e9b7cfbc51372083920e6a9ab133b219193d90f5c371a3957589"} Jul 18 11:28:02.005426 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.005399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" event={"ID":"c5e28c4d-981c-479c-9dc2-3a84f8769386","Type":"ContainerStarted","Data":"50178c3380d74646297ed677ecdfe88c7474a89269f72cfb58cfcfb2fc4ebb86"} Jul 18 11:28:02.005553 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.005456 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" event={"ID":"c5e28c4d-981c-479c-9dc2-3a84f8769386","Type":"ContainerStarted","Data":"83d365771915aafebc07beacdbc28c5994d85a23ebda3e3ddabf56a16b5207ea"} Jul 18 11:28:02.006996 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.006966 2585 generic.go:356] "Generic (PLEG): container finished" podID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerID="2c2499dc48954f7675400d3e983f9ed3528a43ba9d162db2a53c62dc1bc5ffc3" exitCode=0 Jul 18 11:28:02.007092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.007035 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" event={"ID":"07e097f8-f1b8-4c1d-a86a-0728eea9b93d","Type":"ContainerDied","Data":"2c2499dc48954f7675400d3e983f9ed3528a43ba9d162db2a53c62dc1bc5ffc3"} Jul 18 11:28:02.007092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.007076 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" event={"ID":"07e097f8-f1b8-4c1d-a86a-0728eea9b93d","Type":"ContainerStarted","Data":"98117cd8b5178517739c62ad61b28caf376012ac8a70f235cfbbd6f941133308"} Jul 18 11:28:02.200900 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.200856 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8"] Jul 18 11:28:02.204731 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.204708 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.212463 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.212423 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8"] Jul 18 11:28:02.292193 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.292097 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.292193 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.292143 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.292407 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.292273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jwpf\" (UniqueName: \"kubernetes.io/projected/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-kube-api-access-8jwpf\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.393077 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.393036 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jwpf\" (UniqueName: \"kubernetes.io/projected/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-kube-api-access-8jwpf\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.393283 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.393097 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.393283 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.393124 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.393511 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.393492 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.393583 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.393537 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.402834 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.402803 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jwpf\" (UniqueName: \"kubernetes.io/projected/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-kube-api-access-8jwpf\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.551996 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.551907 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:02.727142 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:02.727105 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8"] Jul 18 11:28:02.728597 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:28:02.728570 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bff23c_5278_43fe_bb17_c7ce37bc02d8.slice/crio-a1e10bdc13327889ad5358ee419b2da44e59a929459234f19f3dd13875c81bb5 WatchSource:0}: Error finding container a1e10bdc13327889ad5358ee419b2da44e59a929459234f19f3dd13875c81bb5: Status 404 returned error can't find the container with id a1e10bdc13327889ad5358ee419b2da44e59a929459234f19f3dd13875c81bb5 Jul 18 11:28:03.019303 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.019262 2585 generic.go:356] "Generic (PLEG): container finished" podID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerID="7d635c93af8ab61afc02165139375cc8bd808df03055fd07a44701a535a8fc60" exitCode=0 Jul 18 11:28:03.019712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.019347 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" event={"ID":"8eee5fd2-40ff-4d9e-8701-83005930c682","Type":"ContainerDied","Data":"7d635c93af8ab61afc02165139375cc8bd808df03055fd07a44701a535a8fc60"} Jul 18 11:28:03.020833 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.020810 2585 generic.go:356] "Generic (PLEG): container finished" podID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerID="50178c3380d74646297ed677ecdfe88c7474a89269f72cfb58cfcfb2fc4ebb86" exitCode=0 Jul 18 11:28:03.021027 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.020873 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" event={"ID":"c5e28c4d-981c-479c-9dc2-3a84f8769386","Type":"ContainerDied","Data":"50178c3380d74646297ed677ecdfe88c7474a89269f72cfb58cfcfb2fc4ebb86"} Jul 18 11:28:03.022455 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.022435 2585 generic.go:356] "Generic (PLEG): container finished" podID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerID="b433887869c8a9e564a2e2f49fff3308aa863d62749680f7ab56791b36104132" exitCode=0 Jul 18 11:28:03.022548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.022513 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" event={"ID":"a7bff23c-5278-43fe-bb17-c7ce37bc02d8","Type":"ContainerDied","Data":"b433887869c8a9e564a2e2f49fff3308aa863d62749680f7ab56791b36104132"} Jul 18 11:28:03.022667 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.022554 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" event={"ID":"a7bff23c-5278-43fe-bb17-c7ce37bc02d8","Type":"ContainerStarted","Data":"a1e10bdc13327889ad5358ee419b2da44e59a929459234f19f3dd13875c81bb5"} Jul 18 11:28:03.024644 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.024527 2585 generic.go:356] "Generic (PLEG): container finished" podID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerID="50abdf705e01ae29524cbe18d292de6fdc4732a301740d784c8f59cc67e2bbc7" exitCode=0 Jul 18 11:28:03.024644 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:03.024620 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" event={"ID":"07e097f8-f1b8-4c1d-a86a-0728eea9b93d","Type":"ContainerDied","Data":"50abdf705e01ae29524cbe18d292de6fdc4732a301740d784c8f59cc67e2bbc7"} Jul 18 11:28:04.032387 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.032355 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" event={"ID":"c5e28c4d-981c-479c-9dc2-3a84f8769386","Type":"ContainerStarted","Data":"0c0ef3e8709bc7a5110fb3a76e0048577d6ddf63cdf8ecb5b5f0ad15b6b9b318"} Jul 18 11:28:04.034528 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.034497 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" event={"ID":"a7bff23c-5278-43fe-bb17-c7ce37bc02d8","Type":"ContainerStarted","Data":"2732497e09c79722578b3fa4ac670420b0545dc71e59754304dc118c8600299e"} Jul 18 11:28:04.036989 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.036961 2585 generic.go:356] "Generic (PLEG): container finished" podID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerID="7c7a8b1f3baa5e86f53718d6287bf9104e2a0df1177afefe3cd82b7d73faf134" exitCode=0 Jul 18 11:28:04.037357 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.037328 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" event={"ID":"07e097f8-f1b8-4c1d-a86a-0728eea9b93d","Type":"ContainerDied","Data":"7c7a8b1f3baa5e86f53718d6287bf9104e2a0df1177afefe3cd82b7d73faf134"} Jul 18 11:28:04.183348 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.183318 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:28:04.313303 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.313268 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-util\") pod \"8eee5fd2-40ff-4d9e-8701-83005930c682\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " Jul 18 11:28:04.313486 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.313404 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-948rq\" (UniqueName: \"kubernetes.io/projected/8eee5fd2-40ff-4d9e-8701-83005930c682-kube-api-access-948rq\") pod \"8eee5fd2-40ff-4d9e-8701-83005930c682\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " Jul 18 11:28:04.313486 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.313433 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-bundle\") pod \"8eee5fd2-40ff-4d9e-8701-83005930c682\" (UID: \"8eee5fd2-40ff-4d9e-8701-83005930c682\") " Jul 18 11:28:04.314090 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.314057 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-bundle" (OuterVolumeSpecName: "bundle") pod "8eee5fd2-40ff-4d9e-8701-83005930c682" (UID: "8eee5fd2-40ff-4d9e-8701-83005930c682"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:04.315804 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.315777 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eee5fd2-40ff-4d9e-8701-83005930c682-kube-api-access-948rq" (OuterVolumeSpecName: "kube-api-access-948rq") pod "8eee5fd2-40ff-4d9e-8701-83005930c682" (UID: "8eee5fd2-40ff-4d9e-8701-83005930c682"). InnerVolumeSpecName "kube-api-access-948rq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:28:04.319094 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.319053 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-util" (OuterVolumeSpecName: "util") pod "8eee5fd2-40ff-4d9e-8701-83005930c682" (UID: "8eee5fd2-40ff-4d9e-8701-83005930c682"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:04.414900 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.414856 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-948rq\" (UniqueName: \"kubernetes.io/projected/8eee5fd2-40ff-4d9e-8701-83005930c682-kube-api-access-948rq\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:04.415062 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.414915 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:04.415062 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:04.414929 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8eee5fd2-40ff-4d9e-8701-83005930c682-util\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:05.043753 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.043722 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" Jul 18 11:28:05.044185 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.043722 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd" event={"ID":"8eee5fd2-40ff-4d9e-8701-83005930c682","Type":"ContainerDied","Data":"93e216384c884214c1975b0a73f4e6910af13b0e7e39001df91caf0399162ce6"} Jul 18 11:28:05.044185 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.043848 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93e216384c884214c1975b0a73f4e6910af13b0e7e39001df91caf0399162ce6" Jul 18 11:28:05.045588 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.045562 2585 generic.go:356] "Generic (PLEG): container finished" podID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerID="0c0ef3e8709bc7a5110fb3a76e0048577d6ddf63cdf8ecb5b5f0ad15b6b9b318" exitCode=0 Jul 18 11:28:05.045676 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.045644 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" event={"ID":"c5e28c4d-981c-479c-9dc2-3a84f8769386","Type":"ContainerDied","Data":"0c0ef3e8709bc7a5110fb3a76e0048577d6ddf63cdf8ecb5b5f0ad15b6b9b318"} Jul 18 11:28:05.047869 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.047845 2585 generic.go:356] "Generic (PLEG): container finished" podID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerID="2732497e09c79722578b3fa4ac670420b0545dc71e59754304dc118c8600299e" exitCode=0 Jul 18 11:28:05.048004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.047871 2585 generic.go:356] "Generic (PLEG): container finished" podID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerID="b474e4b2d439503f876e40dff9be9a874da486fde9029d9431ee1adadd778e9e" exitCode=0 Jul 18 11:28:05.048004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.047908 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" event={"ID":"a7bff23c-5278-43fe-bb17-c7ce37bc02d8","Type":"ContainerDied","Data":"2732497e09c79722578b3fa4ac670420b0545dc71e59754304dc118c8600299e"} Jul 18 11:28:05.048004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.047953 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" event={"ID":"a7bff23c-5278-43fe-bb17-c7ce37bc02d8","Type":"ContainerDied","Data":"b474e4b2d439503f876e40dff9be9a874da486fde9029d9431ee1adadd778e9e"} Jul 18 11:28:05.235049 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.235022 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:05.324548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.324452 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvjjg\" (UniqueName: \"kubernetes.io/projected/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-kube-api-access-vvjjg\") pod \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " Jul 18 11:28:05.324695 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.324566 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-bundle\") pod \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " Jul 18 11:28:05.324695 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.324641 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-util\") pod \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\" (UID: \"07e097f8-f1b8-4c1d-a86a-0728eea9b93d\") " Jul 18 11:28:05.325214 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.325181 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-bundle" (OuterVolumeSpecName: "bundle") pod "07e097f8-f1b8-4c1d-a86a-0728eea9b93d" (UID: "07e097f8-f1b8-4c1d-a86a-0728eea9b93d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:05.326665 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.326633 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-kube-api-access-vvjjg" (OuterVolumeSpecName: "kube-api-access-vvjjg") pod "07e097f8-f1b8-4c1d-a86a-0728eea9b93d" (UID: "07e097f8-f1b8-4c1d-a86a-0728eea9b93d"). InnerVolumeSpecName "kube-api-access-vvjjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:28:05.329530 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.329503 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-util" (OuterVolumeSpecName: "util") pod "07e097f8-f1b8-4c1d-a86a-0728eea9b93d" (UID: "07e097f8-f1b8-4c1d-a86a-0728eea9b93d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:05.426194 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.426151 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:05.426194 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.426184 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-util\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:05.426194 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:05.426193 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvjjg\" (UniqueName: \"kubernetes.io/projected/07e097f8-f1b8-4c1d-a86a-0728eea9b93d-kube-api-access-vvjjg\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:06.055549 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.055512 2585 generic.go:356] "Generic (PLEG): container finished" podID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerID="e4a097def27a2ff3f1cc8a7d5b220e9bc8031145ba306f23a523c5fb5f9d6d1b" exitCode=0 Jul 18 11:28:06.056010 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.055611 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" event={"ID":"c5e28c4d-981c-479c-9dc2-3a84f8769386","Type":"ContainerDied","Data":"e4a097def27a2ff3f1cc8a7d5b220e9bc8031145ba306f23a523c5fb5f9d6d1b"} Jul 18 11:28:06.057410 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.057380 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" event={"ID":"07e097f8-f1b8-4c1d-a86a-0728eea9b93d","Type":"ContainerDied","Data":"98117cd8b5178517739c62ad61b28caf376012ac8a70f235cfbbd6f941133308"} Jul 18 11:28:06.057488 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.057412 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb" Jul 18 11:28:06.057488 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.057432 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98117cd8b5178517739c62ad61b28caf376012ac8a70f235cfbbd6f941133308" Jul 18 11:28:06.188414 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.188384 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:06.333789 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.333694 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-util\") pod \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " Jul 18 11:28:06.333789 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.333753 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-bundle\") pod \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " Jul 18 11:28:06.334032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.333793 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jwpf\" (UniqueName: \"kubernetes.io/projected/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-kube-api-access-8jwpf\") pod \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\" (UID: \"a7bff23c-5278-43fe-bb17-c7ce37bc02d8\") " Jul 18 11:28:06.334434 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.334407 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-bundle" (OuterVolumeSpecName: "bundle") pod "a7bff23c-5278-43fe-bb17-c7ce37bc02d8" (UID: "a7bff23c-5278-43fe-bb17-c7ce37bc02d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:06.335943 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.335908 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-kube-api-access-8jwpf" (OuterVolumeSpecName: "kube-api-access-8jwpf") pod "a7bff23c-5278-43fe-bb17-c7ce37bc02d8" (UID: "a7bff23c-5278-43fe-bb17-c7ce37bc02d8"). InnerVolumeSpecName "kube-api-access-8jwpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:28:06.338935 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.338909 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-util" (OuterVolumeSpecName: "util") pod "a7bff23c-5278-43fe-bb17-c7ce37bc02d8" (UID: "a7bff23c-5278-43fe-bb17-c7ce37bc02d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:06.434706 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.434648 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-util\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:06.434706 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.434697 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:06.434706 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:06.434706 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jwpf\" (UniqueName: \"kubernetes.io/projected/a7bff23c-5278-43fe-bb17-c7ce37bc02d8-kube-api-access-8jwpf\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:07.063809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.063781 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" Jul 18 11:28:07.063809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.063777 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8" event={"ID":"a7bff23c-5278-43fe-bb17-c7ce37bc02d8","Type":"ContainerDied","Data":"a1e10bdc13327889ad5358ee419b2da44e59a929459234f19f3dd13875c81bb5"} Jul 18 11:28:07.064259 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.063834 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1e10bdc13327889ad5358ee419b2da44e59a929459234f19f3dd13875c81bb5" Jul 18 11:28:07.196323 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.196295 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:07.344002 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.343902 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsxjd\" (UniqueName: \"kubernetes.io/projected/c5e28c4d-981c-479c-9dc2-3a84f8769386-kube-api-access-rsxjd\") pod \"c5e28c4d-981c-479c-9dc2-3a84f8769386\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " Jul 18 11:28:07.344183 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.344002 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-bundle\") pod \"c5e28c4d-981c-479c-9dc2-3a84f8769386\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " Jul 18 11:28:07.344183 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.344033 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-util\") pod \"c5e28c4d-981c-479c-9dc2-3a84f8769386\" (UID: \"c5e28c4d-981c-479c-9dc2-3a84f8769386\") " Jul 18 11:28:07.344400 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.344377 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-bundle" (OuterVolumeSpecName: "bundle") pod "c5e28c4d-981c-479c-9dc2-3a84f8769386" (UID: "c5e28c4d-981c-479c-9dc2-3a84f8769386"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:07.346170 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.346135 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e28c4d-981c-479c-9dc2-3a84f8769386-kube-api-access-rsxjd" (OuterVolumeSpecName: "kube-api-access-rsxjd") pod "c5e28c4d-981c-479c-9dc2-3a84f8769386" (UID: "c5e28c4d-981c-479c-9dc2-3a84f8769386"). InnerVolumeSpecName "kube-api-access-rsxjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:28:07.348522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.348496 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-util" (OuterVolumeSpecName: "util") pod "c5e28c4d-981c-479c-9dc2-3a84f8769386" (UID: "c5e28c4d-981c-479c-9dc2-3a84f8769386"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:28:07.445668 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.445628 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsxjd\" (UniqueName: \"kubernetes.io/projected/c5e28c4d-981c-479c-9dc2-3a84f8769386-kube-api-access-rsxjd\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:07.445668 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.445660 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:07.445668 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:07.445671 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e28c4d-981c-479c-9dc2-3a84f8769386-util\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:08.071095 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:08.071063 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" Jul 18 11:28:08.071503 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:08.071095 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz" event={"ID":"c5e28c4d-981c-479c-9dc2-3a84f8769386","Type":"ContainerDied","Data":"83d365771915aafebc07beacdbc28c5994d85a23ebda3e3ddabf56a16b5207ea"} Jul 18 11:28:08.071503 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:08.071148 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83d365771915aafebc07beacdbc28c5994d85a23ebda3e3ddabf56a16b5207ea" Jul 18 11:28:13.606301 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:13.606247 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:14.213160 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:14.213123 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:15.849517 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:15.849446 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b766c647-2snv8" podUID="079d31bb-d538-48be-a459-f0821e675178" containerName="console" containerID="cri-o://b9389c64e994b206f87a1608db0ef44aaab5ff659df36e7ff52fac16fb4eabed" gracePeriod=15 Jul 18 11:28:16.111237 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.111148 2585 generic.go:356] "Generic (PLEG): container finished" podID="079d31bb-d538-48be-a459-f0821e675178" containerID="b9389c64e994b206f87a1608db0ef44aaab5ff659df36e7ff52fac16fb4eabed" exitCode=0 Jul 18 11:28:16.111385 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.111270 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b766c647-2snv8" event={"ID":"079d31bb-d538-48be-a459-f0821e675178","Type":"ContainerDied","Data":"b9389c64e994b206f87a1608db0ef44aaab5ff659df36e7ff52fac16fb4eabed"} Jul 18 11:28:16.111385 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.111328 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b766c647-2snv8" event={"ID":"079d31bb-d538-48be-a459-f0821e675178","Type":"ContainerDied","Data":"958959b5f8849c7b580d6a110a9b98c46aa7166c7b85be89737c331251c06920"} Jul 18 11:28:16.111385 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.111347 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="958959b5f8849c7b580d6a110a9b98c46aa7166c7b85be89737c331251c06920" Jul 18 11:28:16.114940 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.114912 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:28:16.224260 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224225 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-console-config\") pod \"079d31bb-d538-48be-a459-f0821e675178\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " Jul 18 11:28:16.224449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224319 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-oauth-config\") pod \"079d31bb-d538-48be-a459-f0821e675178\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " Jul 18 11:28:16.224449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224356 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-oauth-serving-cert\") pod \"079d31bb-d538-48be-a459-f0821e675178\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " Jul 18 11:28:16.224449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224374 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-serving-cert\") pod \"079d31bb-d538-48be-a459-f0821e675178\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " Jul 18 11:28:16.224449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224394 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-service-ca\") pod \"079d31bb-d538-48be-a459-f0821e675178\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " Jul 18 11:28:16.224449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224430 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-trusted-ca-bundle\") pod \"079d31bb-d538-48be-a459-f0821e675178\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " Jul 18 11:28:16.224707 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224474 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb9b4\" (UniqueName: \"kubernetes.io/projected/079d31bb-d538-48be-a459-f0821e675178-kube-api-access-wb9b4\") pod \"079d31bb-d538-48be-a459-f0821e675178\" (UID: \"079d31bb-d538-48be-a459-f0821e675178\") " Jul 18 11:28:16.224781 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224756 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-console-config" (OuterVolumeSpecName: "console-config") pod "079d31bb-d538-48be-a459-f0821e675178" (UID: "079d31bb-d538-48be-a459-f0821e675178"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:28:16.224848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224794 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "079d31bb-d538-48be-a459-f0821e675178" (UID: "079d31bb-d538-48be-a459-f0821e675178"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:28:16.224925 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.224861 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-service-ca" (OuterVolumeSpecName: "service-ca") pod "079d31bb-d538-48be-a459-f0821e675178" (UID: "079d31bb-d538-48be-a459-f0821e675178"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:28:16.225174 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.225149 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "079d31bb-d538-48be-a459-f0821e675178" (UID: "079d31bb-d538-48be-a459-f0821e675178"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:28:16.226778 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.226749 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "079d31bb-d538-48be-a459-f0821e675178" (UID: "079d31bb-d538-48be-a459-f0821e675178"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:28:16.227082 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.227055 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/079d31bb-d538-48be-a459-f0821e675178-kube-api-access-wb9b4" (OuterVolumeSpecName: "kube-api-access-wb9b4") pod "079d31bb-d538-48be-a459-f0821e675178" (UID: "079d31bb-d538-48be-a459-f0821e675178"). InnerVolumeSpecName "kube-api-access-wb9b4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:28:16.227186 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.227088 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "079d31bb-d538-48be-a459-f0821e675178" (UID: "079d31bb-d538-48be-a459-f0821e675178"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:28:16.326074 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.326032 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-oauth-config\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:16.326074 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.326071 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-oauth-serving-cert\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:16.326287 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.326085 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/079d31bb-d538-48be-a459-f0821e675178-console-serving-cert\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:16.326287 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.326100 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:16.326287 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.326111 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-trusted-ca-bundle\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:16.326287 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.326119 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb9b4\" (UniqueName: \"kubernetes.io/projected/079d31bb-d538-48be-a459-f0821e675178-kube-api-access-wb9b4\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:16.326287 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:16.326129 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/079d31bb-d538-48be-a459-f0821e675178-console-config\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:28:17.116219 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:17.116185 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b766c647-2snv8" Jul 18 11:28:17.139369 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:17.139325 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b766c647-2snv8"] Jul 18 11:28:17.141301 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:17.141278 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b766c647-2snv8"] Jul 18 11:28:17.614215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:17.614179 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="079d31bb-d538-48be-a459-f0821e675178" path="/var/lib/kubelet/pods/079d31bb-d538-48be-a459-f0821e675178/volumes" Jul 18 11:28:19.048783 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:19.048750 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:21.051301 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:21.051258 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:21.149812 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:21.149771 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:22.020523 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:22.020484 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:22.833298 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:22.833262 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:25.256673 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:25.256634 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:25.523548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:25.523454 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:28.487082 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:28.487041 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:30.825370 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:30.825325 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:30.931247 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:30.931195 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:31.251439 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:31.251336 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:31.342345 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:31.342310 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:35.028149 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:35.028108 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:35.125121 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:35.125087 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:37.080603 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:37.080569 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:37.204267 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:37.204220 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:37.820679 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:37.820646 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:37.871926 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:37.871866 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:40.920214 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:40.920174 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:42.360130 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:42.360085 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:42.460280 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:42.460238 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:50.421270 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:50.421225 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:50.523335 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:50.523289 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:51.017724 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:51.017683 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:51.076347 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:51.076311 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:51.377943 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:51.377844 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:51.477442 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:51.477401 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:54.405446 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:54.405413 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:54.507878 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:54.507821 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:55.021407 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:55.021375 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:55.118619 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:55.118580 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:56.606828 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:56.606789 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:28:56.665507 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:28:56.665469 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:01.157005 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:01.156919 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:01.259698 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:01.259658 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:03.508561 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:03.508524 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:03.607273 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:03.607237 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:18.018168 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:18.018130 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:18.177960 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:18.177925 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:29.947200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:29.947167 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:30.054042 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.053997 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:30.108082 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.108047 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:30.364380 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364338 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:29:30.364716 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364703 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerName="pull" Jul 18 11:29:30.364760 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364717 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerName="pull" Jul 18 11:29:30.364760 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364727 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerName="util" Jul 18 11:29:30.364760 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364733 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerName="util" Jul 18 11:29:30.364760 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364742 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerName="pull" Jul 18 11:29:30.364760 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364747 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerName="pull" Jul 18 11:29:30.364760 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364758 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerName="pull" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364763 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerName="pull" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364770 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerName="util" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364775 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerName="util" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364784 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerName="util" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364789 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerName="util" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364796 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerName="util" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364802 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerName="util" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364807 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364812 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364818 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364823 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364830 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079d31bb-d538-48be-a459-f0821e675178" containerName="console" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364837 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="079d31bb-d538-48be-a459-f0821e675178" containerName="console" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364843 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerName="pull" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364847 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerName="pull" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364854 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364860 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364868 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364873 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364937 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="07e097f8-f1b8-4c1d-a86a-0728eea9b93d" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364946 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="079d31bb-d538-48be-a459-f0821e675178" containerName="console" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364951 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e28c4d-981c-479c-9dc2-3a84f8769386" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364958 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7bff23c-5278-43fe-bb17-c7ce37bc02d8" containerName="extract" Jul 18 11:29:30.364994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.364964 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eee5fd2-40ff-4d9e-8701-83005930c682" containerName="extract" Jul 18 11:29:30.367923 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.367879 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.370582 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.370550 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 18 11:29:30.370582 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.370566 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 11:29:30.370752 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.370566 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 11:29:30.372271 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.371620 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-799hc\"" Jul 18 11:29:30.374828 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.374804 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:29:30.399061 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.399023 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:29:30.469257 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.469218 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151-config-file\") pod \"limitador-limitador-54d6d988d5-6vwhj\" (UID: \"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.469444 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.469274 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j89cr\" (UniqueName: \"kubernetes.io/projected/1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151-kube-api-access-j89cr\") pod \"limitador-limitador-54d6d988d5-6vwhj\" (UID: \"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.570582 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.570546 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151-config-file\") pod \"limitador-limitador-54d6d988d5-6vwhj\" (UID: \"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.570841 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.570618 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j89cr\" (UniqueName: \"kubernetes.io/projected/1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151-kube-api-access-j89cr\") pod \"limitador-limitador-54d6d988d5-6vwhj\" (UID: \"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.571299 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.571276 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151-config-file\") pod \"limitador-limitador-54d6d988d5-6vwhj\" (UID: \"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.578533 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.578504 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j89cr\" (UniqueName: \"kubernetes.io/projected/1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151-kube-api-access-j89cr\") pod \"limitador-limitador-54d6d988d5-6vwhj\" (UID: \"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.679328 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.679233 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:30.794844 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.794794 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:30.823388 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.823354 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:29:30.825556 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:29:30.825515 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a1cd3f8_425e_4eeb_9b54_9c5a3ed2a151.slice/crio-d0f2d7a6cd032f5879f0c0f709934c7d4177af199f03da256063d775d5f5ebcc WatchSource:0}: Error finding container d0f2d7a6cd032f5879f0c0f709934c7d4177af199f03da256063d775d5f5ebcc: Status 404 returned error can't find the container with id d0f2d7a6cd032f5879f0c0f709934c7d4177af199f03da256063d775d5f5ebcc Jul 18 11:29:30.887673 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:30.887637 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:31.500658 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:31.500606 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" event={"ID":"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151","Type":"ContainerStarted","Data":"d0f2d7a6cd032f5879f0c0f709934c7d4177af199f03da256063d775d5f5ebcc"} Jul 18 11:29:33.518128 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:33.518091 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" event={"ID":"1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151","Type":"ContainerStarted","Data":"1791516d50ccf8e893794a60272d68365849e9a8d7f2b0123a5f98f4e7b203a5"} Jul 18 11:29:33.518519 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:33.518290 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:33.534520 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:33.534463 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" podStartSLOduration=1.036469592 podStartE2EDuration="3.534444961s" totalImagesPullingTime="2.497975369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:29:33.533547082 +0000 UTC m=+568.435311233" watchObservedRunningTime="2026-07-18 11:29:33.534444961 +0000 UTC m=+568.436209098" Jul 18 11:29:44.525313 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:44.525280 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6vwhj" Jul 18 11:29:46.670255 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:46.670218 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:46.769013 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:46.768975 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:47.155594 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:47.155557 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:59.210161 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:59.210119 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:29:59.956757 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:29:59.956716 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:05.524387 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:05.524357 2585 scope.go:117] "RemoveContainer" containerID="b9389c64e994b206f87a1608db0ef44aaab5ff659df36e7ff52fac16fb4eabed" Jul 18 11:30:05.659464 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:05.659427 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:05.757975 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:05.757938 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:05.857994 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:05.857909 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:06.189013 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:06.188922 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:09.699705 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:09.699668 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:10.907474 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:10.907435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:43.421367 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:43.421277 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:43.519400 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:43.519359 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:44.006590 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:44.006553 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:44.330677 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:44.330590 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:44.476403 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:44.476358 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:44.567681 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:44.567650 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:44.738739 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:44.738632 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:30:48.990700 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:48.990667 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:54.195809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:54.195743 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:54.288096 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:54.288062 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:55.335770 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:55.335730 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:55.435533 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:55.435500 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:56.020784 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:56.020744 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:30:56.121720 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:30:56.121686 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:31:01.564439 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:01.564396 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:31:02.175252 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:02.175219 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:31:07.520204 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:07.520162 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:31:08.475018 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:08.474982 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:31:17.826046 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.825998 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7"] Jul 18 11:31:17.829753 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.829729 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:17.833712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.833684 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 11:31:17.833866 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.833684 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 18 11:31:17.833866 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.833685 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ft9rj\"" Jul 18 11:31:17.833866 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.833725 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 11:31:17.839138 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.839112 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7"] Jul 18 11:31:17.858337 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.858303 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:31:17.905660 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.905625 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:17.905859 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.905677 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:17.905859 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.905724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:17.905859 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.905760 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkqmp\" (UniqueName: \"kubernetes.io/projected/f99d2e80-b92f-4963-885f-2d6608964bb7-kube-api-access-zkqmp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:17.905859 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.905830 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f99d2e80-b92f-4963-885f-2d6608964bb7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:17.906050 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:17.905862 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.006835 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.006800 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkqmp\" (UniqueName: \"kubernetes.io/projected/f99d2e80-b92f-4963-885f-2d6608964bb7-kube-api-access-zkqmp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007042 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.006856 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f99d2e80-b92f-4963-885f-2d6608964bb7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007042 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.006921 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007123 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.007087 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007170 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.007155 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007250 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.007211 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007401 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.007353 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007493 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.007473 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.007567 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.007550 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.009263 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.009241 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f99d2e80-b92f-4963-885f-2d6608964bb7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.009548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.009528 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f99d2e80-b92f-4963-885f-2d6608964bb7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.014747 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.014719 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkqmp\" (UniqueName: \"kubernetes.io/projected/f99d2e80-b92f-4963-885f-2d6608964bb7-kube-api-access-zkqmp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2m8x7\" (UID: \"f99d2e80-b92f-4963-885f-2d6608964bb7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.143803 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.143706 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:18.285372 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.285346 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7"] Jul 18 11:31:18.287237 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:31:18.287202 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99d2e80_b92f_4963_885f_2d6608964bb7.slice/crio-f07f8a82db7ef9fc34d4bb51ed703764cf5ab4e3ed0f2aceb4352583af7979eb WatchSource:0}: Error finding container f07f8a82db7ef9fc34d4bb51ed703764cf5ab4e3ed0f2aceb4352583af7979eb: Status 404 returned error can't find the container with id f07f8a82db7ef9fc34d4bb51ed703764cf5ab4e3ed0f2aceb4352583af7979eb Jul 18 11:31:18.288719 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.288701 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:31:18.387682 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:18.387641 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:31:19.085820 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:19.085780 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" event={"ID":"f99d2e80-b92f-4963-885f-2d6608964bb7","Type":"ContainerStarted","Data":"f07f8a82db7ef9fc34d4bb51ed703764cf5ab4e3ed0f2aceb4352583af7979eb"} Jul 18 11:31:20.093189 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:20.093150 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" event={"ID":"f99d2e80-b92f-4963-885f-2d6608964bb7","Type":"ContainerStarted","Data":"5d9b480dff156c2548b842d5adbaeaad1e0d8fa419aa3fa65131961274d0b244"} Jul 18 11:31:20.109937 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:20.109864 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" podStartSLOduration=2.11291668 podStartE2EDuration="3.109846028s" totalImagesPullingTime="996.929348ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:31:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:31:20.108912487 +0000 UTC m=+675.010676626" watchObservedRunningTime="2026-07-18 11:31:20.109846028 +0000 UTC m=+675.011610157" Jul 18 11:31:21.814406 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:21.814367 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:31:23.379587 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:23.379548 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:31:28.144588 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:28.144543 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:28.145071 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:28.144617 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:28.157291 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:28.157260 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:29.150721 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:29.150684 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2m8x7" Jul 18 11:31:32.029607 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.029575 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw"] Jul 18 11:31:32.033166 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.033142 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.035694 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.035672 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 18 11:31:32.048005 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.047970 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw"] Jul 18 11:31:32.064657 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.064617 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:31:32.139747 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.139692 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.139983 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.139787 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.139983 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.139857 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.139983 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.139938 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z4gk\" (UniqueName: \"kubernetes.io/projected/195453e1-3339-4d04-b715-3fa59cc94932-kube-api-access-7z4gk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.139983 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.139971 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.140196 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.139999 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/195453e1-3339-4d04-b715-3fa59cc94932-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.240489 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.240443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.240489 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.240498 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z4gk\" (UniqueName: \"kubernetes.io/projected/195453e1-3339-4d04-b715-3fa59cc94932-kube-api-access-7z4gk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.240733 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.240516 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.240733 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.240536 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/195453e1-3339-4d04-b715-3fa59cc94932-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.240733 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.240570 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.240733 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.240602 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.241021 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.241000 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.241104 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.241005 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.241104 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.241067 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.242896 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.242864 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/195453e1-3339-4d04-b715-3fa59cc94932-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.243206 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.243186 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/195453e1-3339-4d04-b715-3fa59cc94932-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.248256 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.248228 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z4gk\" (UniqueName: \"kubernetes.io/projected/195453e1-3339-4d04-b715-3fa59cc94932-kube-api-access-7z4gk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bt2pw\" (UID: \"195453e1-3339-4d04-b715-3fa59cc94932\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.351071 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.350962 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:32.491328 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:32.491293 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw"] Jul 18 11:31:32.493992 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:31:32.493965 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195453e1_3339_4d04_b715_3fa59cc94932.slice/crio-b83cff09c14795cb9774c0916a2f4b86e53897e48e49a825d728aa911b9336c4 WatchSource:0}: Error finding container b83cff09c14795cb9774c0916a2f4b86e53897e48e49a825d728aa911b9336c4: Status 404 returned error can't find the container with id b83cff09c14795cb9774c0916a2f4b86e53897e48e49a825d728aa911b9336c4 Jul 18 11:31:33.161782 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:33.161741 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" event={"ID":"195453e1-3339-4d04-b715-3fa59cc94932","Type":"ContainerStarted","Data":"ea2032ef262642215111fb29533589c281130a70a76da9c318bba0262de5b027"} Jul 18 11:31:33.162173 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:33.161808 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" event={"ID":"195453e1-3339-4d04-b715-3fa59cc94932","Type":"ContainerStarted","Data":"b83cff09c14795cb9774c0916a2f4b86e53897e48e49a825d728aa911b9336c4"} Jul 18 11:31:33.179967 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:33.179913 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" podStartSLOduration=1.017601067 podStartE2EDuration="1.179896372s" totalImagesPullingTime="162.295305ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:31:33.177306238 +0000 UTC m=+688.079070377" watchObservedRunningTime="2026-07-18 11:31:33.179896372 +0000 UTC m=+688.081660501" Jul 18 11:31:33.979666 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:33.979628 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:31:42.351670 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:42.351633 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:42.352106 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:42.351693 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:42.364173 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:42.364144 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:31:43.230499 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:31:43.230464 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bt2pw" Jul 18 11:32:04.056508 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:32:04.056474 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:32:06.873795 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:32:06.873758 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:32:52.326345 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:32:52.326306 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:32:52.428166 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:32:52.428128 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:32:53.513684 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:32:53.513643 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:32:53.609938 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:32:53.609872 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:33:21.680409 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:33:21.680371 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:33:31.172366 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:33:31.172268 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:33:42.471654 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:33:42.471621 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:34:25.458742 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:34:25.458708 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:34:25.547513 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:34:25.547484 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:34:57.905999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:34:57.905918 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:34:59.165721 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:34:59.165679 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:23.772415 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:23.772378 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:35:34.982489 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:34.982445 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:35:39.916093 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:39.916057 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:41.049317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:41.049267 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:45.874330 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:45.874292 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:45.976778 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:45.976740 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:46.075032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:46.074998 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:46.229368 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:46.229287 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:51.883178 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:51.883139 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:51.939050 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:51.939018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:35:52.175546 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:35:52.175462 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:36:09.574296 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:09.574248 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:36:12.368258 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:12.368221 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:36:12.468470 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:12.468409 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:36:18.975075 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:18.975039 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:36:22.384664 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:22.384629 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:36:25.527808 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:25.527772 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:36:29.474289 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:29.474199 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:36:37.369203 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:37.369160 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:36:46.778627 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:46.778590 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:36:57.973809 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:36:57.973768 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:37:08.468345 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:37:08.468304 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:37:18.480850 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:37:18.480806 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:37:32.982013 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:37:32.981910 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:37:39.772685 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:37:39.772649 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:37:51.366787 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:37:51.366749 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:38:14.785258 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:38:14.785215 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:38:27.504070 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:38:27.504029 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:38:42.978901 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:38:42.978844 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:38:50.577350 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:38:50.577311 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:38:50.628276 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:38:50.628239 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:38:52.474587 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:38:52.474545 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:39:04.380846 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:39:04.380807 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:39:15.072805 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:39:15.072752 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:39:26.881562 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:39:26.881518 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:40:56.643759 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:40:56.643727 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:40:57.729592 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:40:57.729512 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:41:11.976672 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:41:11.976626 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:41:22.577443 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:41:22.577397 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:41:31.182573 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:41:31.182527 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:41:43.071971 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:41:43.071929 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:41:53.284425 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:41:53.284378 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:42:03.886555 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:42:03.886516 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:42:12.185975 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:42:12.185938 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:42:23.183999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:42:23.183960 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:42:35.489779 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:42:35.489701 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:42:43.181350 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:42:43.181306 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:42:53.904937 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:42:53.904898 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:43:15.690138 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:43:15.690095 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:43:24.243648 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:43:24.243612 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:43:24.897293 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:43:24.897252 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:43:36.560315 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:43:36.560275 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:43:36.680034 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:43:36.679994 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:43:47.669524 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:43:47.669487 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:44:02.378676 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:44:02.378585 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:46:03.479760 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:46:03.479717 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:46:05.033486 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:46:05.033454 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:46:06.276354 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:46:06.276304 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:46:21.274937 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:46:21.274898 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:46:30.874627 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:46:30.874584 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:46:40.276600 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:46:40.276553 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:46:53.979141 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:46:53.979096 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:47:03.173795 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:03.173705 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:47:13.583476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:13.583437 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:47:23.373967 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:23.373928 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:47:30.919716 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:30.919678 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:47:30.970249 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:30.970214 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:47:32.874153 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:32.874118 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:47:46.078026 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:46.077986 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:47:55.077855 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:47:55.077816 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:48:04.783043 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:48:04.783002 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:48:14.470681 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:48:14.470637 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:48:27.073449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:48:27.073408 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:48:36.473276 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:48:36.473238 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:48:48.975975 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:48:48.975938 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:49:00.872945 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:00.872901 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:49:13.072335 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:13.072293 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:49:17.606561 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:17.606531 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:17.690913 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:17.690847 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:18.090140 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:18.090101 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:23.642546 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:23.642509 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:23.702203 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:23.702170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:23.894254 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:23.894164 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:24.235045 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:24.234963 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:24.334202 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:24.334170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:26.900921 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:26.900859 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:26.992778 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:26.992741 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:27.067281 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:27.067239 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:27.179014 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:27.178934 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:27.668568 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:27.668531 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:27.769458 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:27.769423 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:34.276846 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:49:34.276801 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:50:03.187634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:03.187540 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:06.379617 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:06.379575 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:50:07.610326 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:07.610287 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:07.773101 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:07.773069 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:08.010020 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:08.009987 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:08.257800 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:08.257770 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:08.355095 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:08.355018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:08.481985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:08.481949 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:08.828851 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:08.828814 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:09.295268 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:09.295216 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:14.516811 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:14.516777 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:14.637981 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:14.637948 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:14.737600 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:14.737565 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:15.142602 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:15.142567 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:15.243047 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:15.243003 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:18.577875 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:18.577829 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:50:22.484437 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:22.484402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:22.584834 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:22.584772 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:35.926952 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:35.926911 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:36.027954 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:36.027920 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:50.137015 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:50.136977 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:53.077826 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:53.077778 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:50:54.623319 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:54.623284 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:54.795180 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:54.795142 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:55.052245 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:55.052211 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:55.284108 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:55.284080 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:55.590759 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:55.590724 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:55.752092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:55.752055 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:58.367476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.367402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:58.451445 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.451412 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:58.457471 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.457388 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq"] Jul 18 11:50:58.461038 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.461014 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.467985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.464465 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-da009c57-serving-cert\"" Jul 18 11:50:58.467985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.464485 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 11:50:58.467985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.464803 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 11:50:58.467985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.466093 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 11:50:58.474931 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.474800 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq"] Jul 18 11:50:58.524038 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.524003 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:58.621829 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.621726 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:58.637175 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.637143 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d6483a-1e92-49e3-b56b-9ad7384f940f-openshift-service-ca\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.637308 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.637198 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdbxk\" (UniqueName: \"kubernetes.io/projected/44d6483a-1e92-49e3-b56b-9ad7384f940f-kube-api-access-mdbxk\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.637308 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.637280 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d6483a-1e92-49e3-b56b-9ad7384f940f-maas-api-tls\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.738611 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.738573 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d6483a-1e92-49e3-b56b-9ad7384f940f-openshift-service-ca\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.738611 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.738617 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdbxk\" (UniqueName: \"kubernetes.io/projected/44d6483a-1e92-49e3-b56b-9ad7384f940f-kube-api-access-mdbxk\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.738838 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.738641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d6483a-1e92-49e3-b56b-9ad7384f940f-maas-api-tls\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.739302 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.739278 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d6483a-1e92-49e3-b56b-9ad7384f940f-openshift-service-ca\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.741171 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.741150 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d6483a-1e92-49e3-b56b-9ad7384f940f-maas-api-tls\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.749975 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.749952 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdbxk\" (UniqueName: \"kubernetes.io/projected/44d6483a-1e92-49e3-b56b-9ad7384f940f-kube-api-access-mdbxk\") pod \"maas-api-e2e-ait-da009c57-5b44d57fbf-596fq\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.778782 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.778742 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:50:58.924126 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.924096 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq"] Jul 18 11:50:58.925354 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:50:58.925322 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d6483a_1e92_49e3_b56b_9ad7384f940f.slice/crio-3e5def147d31bda51e84353457bb6e7c4ec457c303fe3e9f31db1cde5a5f8bfd WatchSource:0}: Error finding container 3e5def147d31bda51e84353457bb6e7c4ec457c303fe3e9f31db1cde5a5f8bfd: Status 404 returned error can't find the container with id 3e5def147d31bda51e84353457bb6e7c4ec457c303fe3e9f31db1cde5a5f8bfd Jul 18 11:50:58.926762 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:58.926744 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:50:59.047564 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:59.047531 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:59.147385 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:59.147353 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:50:59.324193 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:50:59.324160 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" event={"ID":"44d6483a-1e92-49e3-b56b-9ad7384f940f","Type":"ContainerStarted","Data":"3e5def147d31bda51e84353457bb6e7c4ec457c303fe3e9f31db1cde5a5f8bfd"} Jul 18 11:51:01.337122 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:01.337087 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" event={"ID":"44d6483a-1e92-49e3-b56b-9ad7384f940f","Type":"ContainerStarted","Data":"435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256"} Jul 18 11:51:01.337643 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:01.337210 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:51:01.354177 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:01.354123 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" podStartSLOduration=1.102599889 podStartE2EDuration="3.354103478s" totalImagesPullingTime="2.251503589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:51:01.353240142 +0000 UTC m=+1856.255004283" watchObservedRunningTime="2026-07-18 11:51:01.354103478 +0000 UTC m=+1856.255867615" Jul 18 11:51:03.974343 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:03.974299 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:51:07.347941 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:07.347909 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:51:08.646079 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:08.646049 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:09.698499 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:09.698466 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:34.722817 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:34.722776 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq"] Jul 18 11:51:34.723378 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:34.723027 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" podUID="44d6483a-1e92-49e3-b56b-9ad7384f940f" containerName="maas-api" containerID="cri-o://435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256" gracePeriod=30 Jul 18 11:51:34.859114 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:34.859076 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:34.964395 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:34.964367 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:51:35.047030 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.046986 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdbxk\" (UniqueName: \"kubernetes.io/projected/44d6483a-1e92-49e3-b56b-9ad7384f940f-kube-api-access-mdbxk\") pod \"44d6483a-1e92-49e3-b56b-9ad7384f940f\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " Jul 18 11:51:35.047217 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.047068 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d6483a-1e92-49e3-b56b-9ad7384f940f-openshift-service-ca\") pod \"44d6483a-1e92-49e3-b56b-9ad7384f940f\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " Jul 18 11:51:35.047217 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.047141 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d6483a-1e92-49e3-b56b-9ad7384f940f-maas-api-tls\") pod \"44d6483a-1e92-49e3-b56b-9ad7384f940f\" (UID: \"44d6483a-1e92-49e3-b56b-9ad7384f940f\") " Jul 18 11:51:35.047493 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.047462 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d6483a-1e92-49e3-b56b-9ad7384f940f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44d6483a-1e92-49e3-b56b-9ad7384f940f" (UID: "44d6483a-1e92-49e3-b56b-9ad7384f940f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:51:35.049218 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.049185 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d6483a-1e92-49e3-b56b-9ad7384f940f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44d6483a-1e92-49e3-b56b-9ad7384f940f" (UID: "44d6483a-1e92-49e3-b56b-9ad7384f940f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:51:35.049306 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.049221 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d6483a-1e92-49e3-b56b-9ad7384f940f-kube-api-access-mdbxk" (OuterVolumeSpecName: "kube-api-access-mdbxk") pod "44d6483a-1e92-49e3-b56b-9ad7384f940f" (UID: "44d6483a-1e92-49e3-b56b-9ad7384f940f"). InnerVolumeSpecName "kube-api-access-mdbxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:51:35.148030 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.147989 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdbxk\" (UniqueName: \"kubernetes.io/projected/44d6483a-1e92-49e3-b56b-9ad7384f940f-kube-api-access-mdbxk\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:51:35.148030 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.148022 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d6483a-1e92-49e3-b56b-9ad7384f940f-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:51:35.148030 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.148034 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d6483a-1e92-49e3-b56b-9ad7384f940f-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:51:35.519302 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.519270 2585 generic.go:356] "Generic (PLEG): container finished" podID="44d6483a-1e92-49e3-b56b-9ad7384f940f" containerID="435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256" exitCode=0 Jul 18 11:51:35.519491 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.519327 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" Jul 18 11:51:35.519491 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.519369 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" event={"ID":"44d6483a-1e92-49e3-b56b-9ad7384f940f","Type":"ContainerDied","Data":"435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256"} Jul 18 11:51:35.519491 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.519449 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq" event={"ID":"44d6483a-1e92-49e3-b56b-9ad7384f940f","Type":"ContainerDied","Data":"3e5def147d31bda51e84353457bb6e7c4ec457c303fe3e9f31db1cde5a5f8bfd"} Jul 18 11:51:35.519491 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.519471 2585 scope.go:117] "RemoveContainer" containerID="435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256" Jul 18 11:51:35.528634 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.528617 2585 scope.go:117] "RemoveContainer" containerID="435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256" Jul 18 11:51:35.528937 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:51:35.528917 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256\": container with ID starting with 435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256 not found: ID does not exist" containerID="435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256" Jul 18 11:51:35.528992 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.528948 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256"} err="failed to get container status \"435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256\": rpc error: code = NotFound desc = could not find container \"435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256\": container with ID starting with 435cb27f3fdd0bb0e4a6a644f2435877c3680ed654ed7b92b856b4e94d7bf256 not found: ID does not exist" Jul 18 11:51:35.541268 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.541238 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq"] Jul 18 11:51:35.545532 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.545506 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-da009c57-5b44d57fbf-596fq"] Jul 18 11:51:35.614330 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:35.614294 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44d6483a-1e92-49e3-b56b-9ad7384f940f" path="/var/lib/kubelet/pods/44d6483a-1e92-49e3-b56b-9ad7384f940f/volumes" Jul 18 11:51:37.678466 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:37.678413 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:51:39.200008 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:39.199948 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:40.410455 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:40.410417 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:40.666050 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:40.665958 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:40.898999 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:40.898966 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:41.216332 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:41.216285 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:41.317107 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:41.317068 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:43.886893 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.886852 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:43.976412 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.976372 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz"] Jul 18 11:51:43.976950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.976931 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d6483a-1e92-49e3-b56b-9ad7384f940f" containerName="maas-api" Jul 18 11:51:43.976950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.976950 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d6483a-1e92-49e3-b56b-9ad7384f940f" containerName="maas-api" Jul 18 11:51:43.977106 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.977023 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d6483a-1e92-49e3-b56b-9ad7384f940f" containerName="maas-api" Jul 18 11:51:43.979722 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.979691 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:43.980215 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.980198 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:43.983133 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.983111 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 11:51:43.984133 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.984114 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 11:51:43.984243 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.984119 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9573ed82-serving-cert\"" Jul 18 11:51:43.984243 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.984157 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 11:51:43.996310 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:43.996269 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz"] Jul 18 11:51:44.054879 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.054843 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:44.135098 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.135058 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzzdw\" (UniqueName: \"kubernetes.io/projected/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-kube-api-access-bzzdw\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.135310 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.135137 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-maas-api-tls\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.135310 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.135190 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-openshift-service-ca\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.234509 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.234417 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:44.236601 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.236573 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzzdw\" (UniqueName: \"kubernetes.io/projected/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-kube-api-access-bzzdw\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.236693 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.236633 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-maas-api-tls\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.236693 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.236658 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-openshift-service-ca\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.237401 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.237373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-openshift-service-ca\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.239290 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.239266 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-maas-api-tls\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.248810 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.248782 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzzdw\" (UniqueName: \"kubernetes.io/projected/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-kube-api-access-bzzdw\") pod \"maas-api-e2e-derive-9573ed82-77674569d8-x4mzz\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.291535 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.291493 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:44.433739 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.433708 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz"] Jul 18 11:51:44.435809 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:51:44.435774 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1bd3d6c_13cd_4e59_bc29_681a0d68272e.slice/crio-59634ed4c45728201fe28b9583d693bbc383499b6d966d4f7755a2d3060e2d77 WatchSource:0}: Error finding container 59634ed4c45728201fe28b9583d693bbc383499b6d966d4f7755a2d3060e2d77: Status 404 returned error can't find the container with id 59634ed4c45728201fe28b9583d693bbc383499b6d966d4f7755a2d3060e2d77 Jul 18 11:51:44.554289 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.554253 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:44.569192 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.569152 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" event={"ID":"a1bd3d6c-13cd-4e59-bc29-681a0d68272e","Type":"ContainerStarted","Data":"59634ed4c45728201fe28b9583d693bbc383499b6d966d4f7755a2d3060e2d77"} Jul 18 11:51:44.655254 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:44.655220 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:51:45.577106 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:45.577068 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" event={"ID":"a1bd3d6c-13cd-4e59-bc29-681a0d68272e","Type":"ContainerStarted","Data":"61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0"} Jul 18 11:51:45.577520 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:45.577217 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:51:45.598359 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:45.598290 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" podStartSLOduration=2.245751803 podStartE2EDuration="2.598270362s" totalImagesPullingTime="352.518559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:51:45.596374001 +0000 UTC m=+1900.498138156" watchObservedRunningTime="2026-07-18 11:51:45.598270362 +0000 UTC m=+1900.500034500" Jul 18 11:51:48.483037 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:48.483001 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:51:51.590948 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:51:51.590917 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:52:19.211793 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.211749 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz"] Jul 18 11:52:19.212791 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.212726 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" podUID="a1bd3d6c-13cd-4e59-bc29-681a0d68272e" containerName="maas-api" containerID="cri-o://61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0" gracePeriod=30 Jul 18 11:52:19.244805 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.244763 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:19.463472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.463397 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:52:19.551264 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.551226 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-openshift-service-ca\") pod \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " Jul 18 11:52:19.551452 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.551360 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzzdw\" (UniqueName: \"kubernetes.io/projected/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-kube-api-access-bzzdw\") pod \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " Jul 18 11:52:19.551452 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.551391 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-maas-api-tls\") pod \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\" (UID: \"a1bd3d6c-13cd-4e59-bc29-681a0d68272e\") " Jul 18 11:52:19.551701 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.551672 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1bd3d6c-13cd-4e59-bc29-681a0d68272e" (UID: "a1bd3d6c-13cd-4e59-bc29-681a0d68272e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:19.553652 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.553622 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-kube-api-access-bzzdw" (OuterVolumeSpecName: "kube-api-access-bzzdw") pod "a1bd3d6c-13cd-4e59-bc29-681a0d68272e" (UID: "a1bd3d6c-13cd-4e59-bc29-681a0d68272e"). InnerVolumeSpecName "kube-api-access-bzzdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:52:19.553652 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.553634 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1bd3d6c-13cd-4e59-bc29-681a0d68272e" (UID: "a1bd3d6c-13cd-4e59-bc29-681a0d68272e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:52:19.652399 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.652354 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzzdw\" (UniqueName: \"kubernetes.io/projected/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-kube-api-access-bzzdw\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:52:19.652399 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.652391 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:52:19.652399 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.652400 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1bd3d6c-13cd-4e59-bc29-681a0d68272e-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:52:19.764253 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.764221 2585 generic.go:356] "Generic (PLEG): container finished" podID="a1bd3d6c-13cd-4e59-bc29-681a0d68272e" containerID="61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0" exitCode=0 Jul 18 11:52:19.764449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.764271 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" Jul 18 11:52:19.764449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.764305 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" event={"ID":"a1bd3d6c-13cd-4e59-bc29-681a0d68272e","Type":"ContainerDied","Data":"61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0"} Jul 18 11:52:19.764449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.764372 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz" event={"ID":"a1bd3d6c-13cd-4e59-bc29-681a0d68272e","Type":"ContainerDied","Data":"59634ed4c45728201fe28b9583d693bbc383499b6d966d4f7755a2d3060e2d77"} Jul 18 11:52:19.764449 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.764401 2585 scope.go:117] "RemoveContainer" containerID="61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0" Jul 18 11:52:19.773742 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.773720 2585 scope.go:117] "RemoveContainer" containerID="61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0" Jul 18 11:52:19.774064 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:52:19.774041 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0\": container with ID starting with 61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0 not found: ID does not exist" containerID="61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0" Jul 18 11:52:19.774159 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.774073 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0"} err="failed to get container status \"61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0\": rpc error: code = NotFound desc = could not find container \"61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0\": container with ID starting with 61f92333c7c3af6a8a954e97be747809f325d0487a27abf128866a7aad14abd0 not found: ID does not exist" Jul 18 11:52:19.781976 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.781946 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz"] Jul 18 11:52:19.786315 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:19.786281 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9573ed82-77674569d8-x4mzz"] Jul 18 11:52:21.614652 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:21.614608 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1bd3d6c-13cd-4e59-bc29-681a0d68272e" path="/var/lib/kubelet/pods/a1bd3d6c-13cd-4e59-bc29-681a0d68272e/volumes" Jul 18 11:52:22.172172 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:22.172130 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:52:23.887434 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:23.887399 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:24.047544 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:24.047514 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:24.294133 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:24.294098 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:24.792266 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:24.792232 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:24.899275 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:24.899241 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:25.033835 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:25.033795 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:26.497018 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:26.496985 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:32.319382 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:32.319347 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:32.424432 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:32.424395 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:32.519616 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:32.519577 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:32.619865 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:32.619775 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:33.047478 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:33.047441 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:52:59.057711 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:52:59.057621 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:05.314542 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:05.314506 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:05.420232 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:05.420193 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:05.533232 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:05.533190 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:21.462937 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:21.462903 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:27.011675 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:27.011636 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:27.114087 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:27.114042 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:27.215829 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:27.215794 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:37.773075 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:37.773034 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:38.571673 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:38.571637 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:44.919175 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:44.919135 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:45.032604 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:45.032557 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:45.121470 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:45.121428 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:45.220168 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:45.220078 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:46.025457 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:46.025421 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:46.120068 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:46.120029 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:46.221117 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:46.221081 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:47.921171 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:47.921136 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:48.579763 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:48.579731 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:53:53.703203 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:53.703159 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:53.794794 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:53.794748 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:53:54.325489 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:53:54.325454 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:00.270239 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:00.270196 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:00.289150 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:00.289117 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:54:00.373424 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:00.373394 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:00.521559 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:00.521475 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:00.851836 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:00.851740 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:01.032173 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:01.032137 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:04.354466 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.354416 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:04.450291 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.450228 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:04.595000 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.594960 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6"] Jul 18 11:54:04.596008 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.595983 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1bd3d6c-13cd-4e59-bc29-681a0d68272e" containerName="maas-api" Jul 18 11:54:04.596008 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.596009 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1bd3d6c-13cd-4e59-bc29-681a0d68272e" containerName="maas-api" Jul 18 11:54:04.596209 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.596164 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1bd3d6c-13cd-4e59-bc29-681a0d68272e" containerName="maas-api" Jul 18 11:54:04.602185 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.602148 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.605339 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.605238 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a8e6b1ef-serving-cert\"" Jul 18 11:54:04.605339 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.605239 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 11:54:04.605743 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.605641 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 11:54:04.605743 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.605668 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 11:54:04.607317 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.607292 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6"] Jul 18 11:54:04.631443 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.631408 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:04.687575 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.687540 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.687793 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.687665 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aaef9d9-ed65-4de8-9762-c036d43e22b1-openshift-service-ca\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.687793 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.687769 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knbqr\" (UniqueName: \"kubernetes.io/projected/8aaef9d9-ed65-4de8-9762-c036d43e22b1-kube-api-access-knbqr\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.788798 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.788753 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aaef9d9-ed65-4de8-9762-c036d43e22b1-openshift-service-ca\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.789037 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.788843 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knbqr\" (UniqueName: \"kubernetes.io/projected/8aaef9d9-ed65-4de8-9762-c036d43e22b1-kube-api-access-knbqr\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.789037 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.788918 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.789124 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:54:04.789058 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-serving-cert: secret "maas-api-e2e-mt-a8e6b1ef-serving-cert" not found Jul 18 11:54:04.789164 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:54:04.789137 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls podName:8aaef9d9-ed65-4de8-9762-c036d43e22b1 nodeName:}" failed. No retries permitted until 2026-07-18 11:54:05.289119907 +0000 UTC m=+2040.190884023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls") pod "maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" (UID: "8aaef9d9-ed65-4de8-9762-c036d43e22b1") : secret "maas-api-e2e-mt-a8e6b1ef-serving-cert" not found Jul 18 11:54:04.789424 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.789406 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aaef9d9-ed65-4de8-9762-c036d43e22b1-openshift-service-ca\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:04.804008 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:04.803971 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knbqr\" (UniqueName: \"kubernetes.io/projected/8aaef9d9-ed65-4de8-9762-c036d43e22b1-kube-api-access-knbqr\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:05.140016 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:05.139979 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:05.234949 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:05.234908 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:05.293344 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:05.293302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:05.296472 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:05.296442 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls\") pod \"maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:05.518154 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:05.518124 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 11:54:05.525728 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:05.525692 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:05.668631 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:05.668595 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6"] Jul 18 11:54:05.669799 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:54:05.669774 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aaef9d9_ed65_4de8_9762_c036d43e22b1.slice/crio-cbdf707c6ae0cd7d42e7cf4a89c4fff1781fa93f7332e166e9f8e24af2ef7039 WatchSource:0}: Error finding container cbdf707c6ae0cd7d42e7cf4a89c4fff1781fa93f7332e166e9f8e24af2ef7039: Status 404 returned error can't find the container with id cbdf707c6ae0cd7d42e7cf4a89c4fff1781fa93f7332e166e9f8e24af2ef7039 Jul 18 11:54:06.325654 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:06.325617 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" event={"ID":"8aaef9d9-ed65-4de8-9762-c036d43e22b1","Type":"ContainerStarted","Data":"6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5"} Jul 18 11:54:06.325845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:06.325670 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" event={"ID":"8aaef9d9-ed65-4de8-9762-c036d43e22b1","Type":"ContainerStarted","Data":"cbdf707c6ae0cd7d42e7cf4a89c4fff1781fa93f7332e166e9f8e24af2ef7039"} Jul 18 11:54:06.325845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:06.325756 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:06.346130 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:06.346079 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" podStartSLOduration=1.9111275650000001 podStartE2EDuration="2.346059685s" totalImagesPullingTime="434.93212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:54:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:54:06.344164574 +0000 UTC m=+2041.245928714" watchObservedRunningTime="2026-07-18 11:54:06.346059685 +0000 UTC m=+2041.247823853" Jul 18 11:54:08.757132 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:08.757091 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:09.414283 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:09.414247 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:09.511550 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:09.511516 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:09.875747 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:09.875711 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:12.336959 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:12.336926 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:14.881821 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:14.881780 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:54:15.629619 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:15.629579 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:15.730857 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:15.730819 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:23.844272 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:23.844231 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:26.478950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.478905 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6"] Jul 18 11:54:26.479387 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.479126 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" podUID="8aaef9d9-ed65-4de8-9762-c036d43e22b1" containerName="maas-api" containerID="cri-o://6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5" gracePeriod=30 Jul 18 11:54:26.532003 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.531972 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:26.731476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.731398 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:26.885290 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.885255 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knbqr\" (UniqueName: \"kubernetes.io/projected/8aaef9d9-ed65-4de8-9762-c036d43e22b1-kube-api-access-knbqr\") pod \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " Jul 18 11:54:26.885482 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.885361 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls\") pod \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " Jul 18 11:54:26.885559 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.885537 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aaef9d9-ed65-4de8-9762-c036d43e22b1-openshift-service-ca\") pod \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\" (UID: \"8aaef9d9-ed65-4de8-9762-c036d43e22b1\") " Jul 18 11:54:26.885932 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.885879 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aaef9d9-ed65-4de8-9762-c036d43e22b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8aaef9d9-ed65-4de8-9762-c036d43e22b1" (UID: "8aaef9d9-ed65-4de8-9762-c036d43e22b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:54:26.887453 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.887430 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8aaef9d9-ed65-4de8-9762-c036d43e22b1" (UID: "8aaef9d9-ed65-4de8-9762-c036d43e22b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:54:26.887620 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.887600 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aaef9d9-ed65-4de8-9762-c036d43e22b1-kube-api-access-knbqr" (OuterVolumeSpecName: "kube-api-access-knbqr") pod "8aaef9d9-ed65-4de8-9762-c036d43e22b1" (UID: "8aaef9d9-ed65-4de8-9762-c036d43e22b1"). InnerVolumeSpecName "kube-api-access-knbqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:54:26.986452 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.986348 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knbqr\" (UniqueName: \"kubernetes.io/projected/8aaef9d9-ed65-4de8-9762-c036d43e22b1-kube-api-access-knbqr\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:54:26.986452 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.986392 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aaef9d9-ed65-4de8-9762-c036d43e22b1-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:54:26.986452 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:26.986406 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aaef9d9-ed65-4de8-9762-c036d43e22b1-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:54:27.440477 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.440428 2585 generic.go:356] "Generic (PLEG): container finished" podID="8aaef9d9-ed65-4de8-9762-c036d43e22b1" containerID="6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5" exitCode=0 Jul 18 11:54:27.440477 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.440478 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" Jul 18 11:54:27.440771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.440526 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" event={"ID":"8aaef9d9-ed65-4de8-9762-c036d43e22b1","Type":"ContainerDied","Data":"6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5"} Jul 18 11:54:27.440771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.440610 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6" event={"ID":"8aaef9d9-ed65-4de8-9762-c036d43e22b1","Type":"ContainerDied","Data":"cbdf707c6ae0cd7d42e7cf4a89c4fff1781fa93f7332e166e9f8e24af2ef7039"} Jul 18 11:54:27.440771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.440640 2585 scope.go:117] "RemoveContainer" containerID="6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5" Jul 18 11:54:27.450832 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.450808 2585 scope.go:117] "RemoveContainer" containerID="6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5" Jul 18 11:54:27.451196 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:54:27.451172 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5\": container with ID starting with 6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5 not found: ID does not exist" containerID="6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5" Jul 18 11:54:27.451253 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.451207 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5"} err="failed to get container status \"6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5\": rpc error: code = NotFound desc = could not find container \"6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5\": container with ID starting with 6593a9ea3aabeaac90aed4faf466a4fe25993d2590a786e70df13da7a12d5bb5 not found: ID does not exist" Jul 18 11:54:27.463353 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.463317 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6"] Jul 18 11:54:27.469577 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.469543 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8e6b1ef-5bbc85cb85-7gmq6"] Jul 18 11:54:27.614222 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:27.614187 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aaef9d9-ed65-4de8-9762-c036d43e22b1" path="/var/lib/kubelet/pods/8aaef9d9-ed65-4de8-9762-c036d43e22b1/volumes" Jul 18 11:54:29.393757 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:29.393716 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:54:31.076585 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:31.076550 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:31.516153 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:31.516121 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:37.235134 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:37.235096 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:37.292950 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:37.292914 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:37.505039 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:37.505003 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:37.746463 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:37.746425 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:37.873694 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:37.873599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:37.971158 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:37.971122 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:38.072316 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:38.072277 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:39.418444 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:39.418404 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:43.607662 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:43.607621 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:43.718686 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:43.718641 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:43.830541 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:43.830502 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:43.989572 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:43.989479 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:49.604419 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:49.604377 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:49.706176 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:49.706141 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:56.482848 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:56.482815 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:56.798595 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:56.798557 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:57.062563 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:57.062481 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:59.516582 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.516546 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:59.603358 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.603315 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq"] Jul 18 11:54:59.603812 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.603791 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:59.603917 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.603905 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aaef9d9-ed65-4de8-9762-c036d43e22b1" containerName="maas-api" Jul 18 11:54:59.603985 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.603924 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aaef9d9-ed65-4de8-9762-c036d43e22b1" containerName="maas-api" Jul 18 11:54:59.604046 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.604030 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aaef9d9-ed65-4de8-9762-c036d43e22b1" containerName="maas-api" Jul 18 11:54:59.607488 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.607467 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.610074 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.610052 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ed7924cb-serving-cert\"" Jul 18 11:54:59.610512 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.610474 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 11:54:59.610639 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.610530 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 11:54:59.611856 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.611831 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 11:54:59.618845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.618822 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq"] Jul 18 11:54:59.675100 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.675059 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:59.775216 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.775076 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:54:59.775427 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.775408 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8633536a-561f-4c4a-bccf-33ea92f3df9c-openshift-service-ca\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.775479 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.775463 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8633536a-561f-4c4a-bccf-33ea92f3df9c-maas-api-tls\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.775571 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.775546 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqpqd\" (UniqueName: \"kubernetes.io/projected/8633536a-561f-4c4a-bccf-33ea92f3df9c-kube-api-access-fqpqd\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.876257 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.876221 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8633536a-561f-4c4a-bccf-33ea92f3df9c-maas-api-tls\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.876460 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.876279 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqpqd\" (UniqueName: \"kubernetes.io/projected/8633536a-561f-4c4a-bccf-33ea92f3df9c-kube-api-access-fqpqd\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.876460 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.876337 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8633536a-561f-4c4a-bccf-33ea92f3df9c-openshift-service-ca\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.877058 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.877034 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8633536a-561f-4c4a-bccf-33ea92f3df9c-openshift-service-ca\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.878693 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.878666 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8633536a-561f-4c4a-bccf-33ea92f3df9c-maas-api-tls\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.885200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.885177 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqpqd\" (UniqueName: \"kubernetes.io/projected/8633536a-561f-4c4a-bccf-33ea92f3df9c-kube-api-access-fqpqd\") pod \"maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:54:59.921295 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:54:59.921253 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:55:00.067482 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:00.067450 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq"] Jul 18 11:55:00.069334 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:55:00.069302 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8633536a_561f_4c4a_bccf_33ea92f3df9c.slice/crio-b02d7028994bfb80dfbc2749fa654f1048ac3cfd3185b526f94e29663e6abbe8 WatchSource:0}: Error finding container b02d7028994bfb80dfbc2749fa654f1048ac3cfd3185b526f94e29663e6abbe8: Status 404 returned error can't find the container with id b02d7028994bfb80dfbc2749fa654f1048ac3cfd3185b526f94e29663e6abbe8 Jul 18 11:55:00.119503 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:00.119463 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:00.219417 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:00.219383 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:00.625011 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:00.624974 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" event={"ID":"8633536a-561f-4c4a-bccf-33ea92f3df9c","Type":"ContainerStarted","Data":"7b419ad457b702fdc762f8a3664d601274188b65149fe3036a89b153356462a9"} Jul 18 11:55:00.625419 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:00.625026 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" event={"ID":"8633536a-561f-4c4a-bccf-33ea92f3df9c","Type":"ContainerStarted","Data":"b02d7028994bfb80dfbc2749fa654f1048ac3cfd3185b526f94e29663e6abbe8"} Jul 18 11:55:00.625419 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:00.625101 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:55:00.643851 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:00.643796 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" podStartSLOduration=1.245318296 podStartE2EDuration="1.643778005s" totalImagesPullingTime="398.459709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:54:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:55:00.642056267 +0000 UTC m=+2095.543820406" watchObservedRunningTime="2026-07-18 11:55:00.643778005 +0000 UTC m=+2095.545542177" Jul 18 11:55:04.092303 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:04.092263 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:04.785543 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:04.785507 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:55:06.635512 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:06.635473 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:55:09.254936 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:09.254879 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:16.181372 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:16.181330 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:55:23.228469 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:23.228434 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:23.328200 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:23.328167 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:28.986383 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:28.986339 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:55:32.806990 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:32.806951 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:32.902205 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:32.902170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:46.280902 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:46.280829 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:55:47.683922 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:47.683868 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq"] Jul 18 11:55:47.684493 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:47.684138 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" podUID="8633536a-561f-4c4a-bccf-33ea92f3df9c" containerName="maas-api" containerID="cri-o://7b419ad457b702fdc762f8a3664d601274188b65149fe3036a89b153356462a9" gracePeriod=30 Jul 18 11:55:47.716193 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:47.716155 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:47.887782 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:47.887750 2585 generic.go:356] "Generic (PLEG): container finished" podID="8633536a-561f-4c4a-bccf-33ea92f3df9c" containerID="7b419ad457b702fdc762f8a3664d601274188b65149fe3036a89b153356462a9" exitCode=0 Jul 18 11:55:47.887971 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:47.887822 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" event={"ID":"8633536a-561f-4c4a-bccf-33ea92f3df9c","Type":"ContainerDied","Data":"7b419ad457b702fdc762f8a3664d601274188b65149fe3036a89b153356462a9"} Jul 18 11:55:47.944792 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:47.944717 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:55:48.106826 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.106783 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8633536a-561f-4c4a-bccf-33ea92f3df9c-openshift-service-ca\") pod \"8633536a-561f-4c4a-bccf-33ea92f3df9c\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " Jul 18 11:55:48.107032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.106860 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8633536a-561f-4c4a-bccf-33ea92f3df9c-maas-api-tls\") pod \"8633536a-561f-4c4a-bccf-33ea92f3df9c\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " Jul 18 11:55:48.107032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.106993 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqpqd\" (UniqueName: \"kubernetes.io/projected/8633536a-561f-4c4a-bccf-33ea92f3df9c-kube-api-access-fqpqd\") pod \"8633536a-561f-4c4a-bccf-33ea92f3df9c\" (UID: \"8633536a-561f-4c4a-bccf-33ea92f3df9c\") " Jul 18 11:55:48.107256 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.107226 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8633536a-561f-4c4a-bccf-33ea92f3df9c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8633536a-561f-4c4a-bccf-33ea92f3df9c" (UID: "8633536a-561f-4c4a-bccf-33ea92f3df9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:55:48.109044 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.109014 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8633536a-561f-4c4a-bccf-33ea92f3df9c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8633536a-561f-4c4a-bccf-33ea92f3df9c" (UID: "8633536a-561f-4c4a-bccf-33ea92f3df9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:55:48.109044 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.109036 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8633536a-561f-4c4a-bccf-33ea92f3df9c-kube-api-access-fqpqd" (OuterVolumeSpecName: "kube-api-access-fqpqd") pod "8633536a-561f-4c4a-bccf-33ea92f3df9c" (UID: "8633536a-561f-4c4a-bccf-33ea92f3df9c"). InnerVolumeSpecName "kube-api-access-fqpqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:55:48.208012 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.207921 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqpqd\" (UniqueName: \"kubernetes.io/projected/8633536a-561f-4c4a-bccf-33ea92f3df9c-kube-api-access-fqpqd\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.208012 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.207955 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8633536a-561f-4c4a-bccf-33ea92f3df9c-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.208012 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.207968 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8633536a-561f-4c4a-bccf-33ea92f3df9c-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.895512 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.895481 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" Jul 18 11:55:48.895982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.895522 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq" event={"ID":"8633536a-561f-4c4a-bccf-33ea92f3df9c","Type":"ContainerDied","Data":"b02d7028994bfb80dfbc2749fa654f1048ac3cfd3185b526f94e29663e6abbe8"} Jul 18 11:55:48.895982 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.895581 2585 scope.go:117] "RemoveContainer" containerID="7b419ad457b702fdc762f8a3664d601274188b65149fe3036a89b153356462a9" Jul 18 11:55:48.917587 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.917546 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq"] Jul 18 11:55:48.924047 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:48.924012 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ed7924cb-5d46bbcd6c-j6crq"] Jul 18 11:55:49.614786 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:49.614749 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8633536a-561f-4c4a-bccf-33ea92f3df9c" path="/var/lib/kubelet/pods/8633536a-561f-4c4a-bccf-33ea92f3df9c/volumes" Jul 18 11:55:52.350453 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:52.350412 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:54.688222 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:54.688191 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:55:56.980590 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:56.980555 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:55:59.752645 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:55:59.752606 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:35.881574 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:35.881539 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:36.945581 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:36.945528 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:42.703771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:42.703722 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:42.803052 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:42.803016 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:43.037154 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:43.037114 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:43.233030 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:43.232985 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:44.295670 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:44.295637 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:44.394266 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:44.394231 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:44.537251 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:44.537215 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:45.266087 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:45.266050 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:45.365070 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:45.365038 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:45.465108 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:45.465069 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:46.683471 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:46.683428 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:56:46.921234 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:46.921191 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:52.855258 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:52.855224 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:52.919142 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:52.919106 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:53.550257 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:53.550219 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:56:58.881807 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:56:58.881769 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:57:00.068004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:00.067967 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:00.170948 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:00.170911 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:00.268143 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:00.268099 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:10.794299 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:10.794261 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:11.170953 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:11.170847 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:13.817774 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:13.817739 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:16.690213 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:16.690169 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:32.133345 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:32.133258 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:38.713866 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:38.713829 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:38.814430 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:38.814392 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:39.074572 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:39.074477 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:39.391580 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:39.391495 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:39.740080 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:39.739977 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:42.445841 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.445798 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:42.545725 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.545683 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:42.599440 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.599400 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg"] Jul 18 11:57:42.599806 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.599793 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8633536a-561f-4c4a-bccf-33ea92f3df9c" containerName="maas-api" Jul 18 11:57:42.599851 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.599807 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8633536a-561f-4c4a-bccf-33ea92f3df9c" containerName="maas-api" Jul 18 11:57:42.599919 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.599895 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8633536a-561f-4c4a-bccf-33ea92f3df9c" containerName="maas-api" Jul 18 11:57:42.602826 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.602807 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.605464 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.605439 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 11:57:42.605607 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.605494 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 11:57:42.606650 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.606630 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 11:57:42.606757 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.606629 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-f7319d-serving-cert\"" Jul 18 11:57:42.615266 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.615234 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg"] Jul 18 11:57:42.678500 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.678465 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:42.747661 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.747575 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.747661 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.747613 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.747866 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.747759 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjrmf\" (UniqueName: \"kubernetes.io/projected/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-kube-api-access-zjrmf\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.849253 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.849213 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjrmf\" (UniqueName: \"kubernetes.io/projected/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-kube-api-access-zjrmf\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.849253 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.849253 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.849475 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.849274 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.849475 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:57:42.849419 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-serving-cert: secret "maas-api-e2e-inf-a-f7319d-serving-cert" not found Jul 18 11:57:42.849475 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:57:42.849476 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls podName:0e043d14-df0f-4f40-9ac5-69c2e93cfcbf nodeName:}" failed. No retries permitted until 2026-07-18 11:57:43.34945745 +0000 UTC m=+2258.251221566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls") pod "maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" (UID: "0e043d14-df0f-4f40-9ac5-69c2e93cfcbf") : secret "maas-api-e2e-inf-a-f7319d-serving-cert" not found Jul 18 11:57:42.850047 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.850028 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:42.858558 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:42.858521 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjrmf\" (UniqueName: \"kubernetes.io/projected/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-kube-api-access-zjrmf\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:43.035527 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.035491 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:43.211078 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.211043 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:43.314583 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.314488 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:43.353683 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.353648 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:43.356234 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.356201 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls\") pod \"maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:43.513845 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.513798 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:43.654188 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.654163 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg"] Jul 18 11:57:43.656424 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:57:43.656400 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e043d14_df0f_4f40_9ac5_69c2e93cfcbf.slice/crio-a9ab30bcbdb3f96a241d0fcc6756cdcc774d98fc2778d24db5c7e20667a099c2 WatchSource:0}: Error finding container a9ab30bcbdb3f96a241d0fcc6756cdcc774d98fc2778d24db5c7e20667a099c2: Status 404 returned error can't find the container with id a9ab30bcbdb3f96a241d0fcc6756cdcc774d98fc2778d24db5c7e20667a099c2 Jul 18 11:57:43.657714 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:43.657697 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:57:44.498259 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:44.498169 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" event={"ID":"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf","Type":"ContainerStarted","Data":"b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854"} Jul 18 11:57:44.498259 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:44.498220 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" event={"ID":"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf","Type":"ContainerStarted","Data":"a9ab30bcbdb3f96a241d0fcc6756cdcc774d98fc2778d24db5c7e20667a099c2"} Jul 18 11:57:44.498461 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:44.498305 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:44.516141 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:44.516082 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" podStartSLOduration=1.981593594 podStartE2EDuration="2.516064688s" totalImagesPullingTime="534.471094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:57:44.513758161 +0000 UTC m=+2259.415522300" watchObservedRunningTime="2026-07-18 11:57:44.516064688 +0000 UTC m=+2259.417828872" Jul 18 11:57:46.970451 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:46.970412 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:47.328439 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:47.328408 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:47.643655 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:47.643571 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:47.875079 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:47.875041 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:57:47.949584 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:47.949499 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:50.509659 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:50.509627 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:57:50.851409 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:50.851328 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:50.945283 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:50.945240 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:51.109476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.109395 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l"] Jul 18 11:57:51.113967 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.113940 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.116771 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.116745 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3a9e62-serving-cert\"" Jul 18 11:57:51.122192 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.122145 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l"] Jul 18 11:57:51.145305 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.145270 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:51.221322 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.221277 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.221519 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.221349 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh4k6\" (UniqueName: \"kubernetes.io/projected/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-kube-api-access-qh4k6\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.221519 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.221425 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.322609 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.322575 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh4k6\" (UniqueName: \"kubernetes.io/projected/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-kube-api-access-qh4k6\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.322795 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.322644 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.322795 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.322688 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.323460 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.323440 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.325334 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.325312 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-maas-api-tls\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.344363 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.344322 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh4k6\" (UniqueName: \"kubernetes.io/projected/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-kube-api-access-qh4k6\") pod \"maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.364754 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.364659 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:51.427825 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.427779 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:51.571342 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.571310 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l"] Jul 18 11:57:51.573508 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:57:51.573482 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1620dbdb_1cc2_4f32_a2df_d4593383b2a9.slice/crio-e115eed01873097009ee93439ea417cedabd4814f7976916ae1e1e483f4155e7 WatchSource:0}: Error finding container e115eed01873097009ee93439ea417cedabd4814f7976916ae1e1e483f4155e7: Status 404 returned error can't find the container with id e115eed01873097009ee93439ea417cedabd4814f7976916ae1e1e483f4155e7 Jul 18 11:57:51.683680 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.683587 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:51.781049 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:51.781017 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:52.544273 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:52.544233 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" event={"ID":"1620dbdb-1cc2-4f32-a2df-d4593383b2a9","Type":"ContainerStarted","Data":"91064e12e206ea5a90370215238e269f1b531cd28ec5d9454b2f820a69c562ee"} Jul 18 11:57:52.544476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:52.544318 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" event={"ID":"1620dbdb-1cc2-4f32-a2df-d4593383b2a9","Type":"ContainerStarted","Data":"e115eed01873097009ee93439ea417cedabd4814f7976916ae1e1e483f4155e7"} Jul 18 11:57:52.544476 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:52.544343 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:57:52.561663 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:52.561604 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" podStartSLOduration=1.260950962 podStartE2EDuration="1.561584063s" totalImagesPullingTime="300.633101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:57:52.55938949 +0000 UTC m=+2267.461153666" watchObservedRunningTime="2026-07-18 11:57:52.561584063 +0000 UTC m=+2267.463348201" Jul 18 11:57:55.264325 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:55.264286 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:57:58.555866 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:57:58.555830 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:58:03.476405 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:03.476362 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:58:04.117844 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:04.117810 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:58:04.218080 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:04.218039 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:58:12.449425 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:12.449385 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:58:12.552100 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:12.552063 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:58:16.081687 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:16.081648 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:58:26.773745 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:26.773708 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:58:51.957186 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:51.957115 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:58:57.675712 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:58:57.675618 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:59:09.480474 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:09.480404 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:10.435825 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:10.435783 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:11.070158 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:11.070120 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:11.176567 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:11.176532 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:16.375226 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:16.375175 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:59:20.982770 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:20.982732 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg"] Jul 18 11:59:20.983548 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:20.983501 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" podUID="0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" containerName="maas-api" containerID="cri-o://b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854" gracePeriod=30 Jul 18 11:59:21.006116 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.006083 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:21.236807 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.236747 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:59:21.336481 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.336443 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-openshift-service-ca\") pod \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " Jul 18 11:59:21.336670 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.336502 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls\") pod \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " Jul 18 11:59:21.336670 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.336522 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjrmf\" (UniqueName: \"kubernetes.io/projected/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-kube-api-access-zjrmf\") pod \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\" (UID: \"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf\") " Jul 18 11:59:21.336972 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.336939 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" (UID: "0e043d14-df0f-4f40-9ac5-69c2e93cfcbf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:59:21.338657 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.338631 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" (UID: "0e043d14-df0f-4f40-9ac5-69c2e93cfcbf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:59:21.338754 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.338677 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-kube-api-access-zjrmf" (OuterVolumeSpecName: "kube-api-access-zjrmf") pod "0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" (UID: "0e043d14-df0f-4f40-9ac5-69c2e93cfcbf"). InnerVolumeSpecName "kube-api-access-zjrmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:59:21.437623 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.437590 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:59:21.437623 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.437617 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:59:21.437623 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:21.437629 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjrmf\" (UniqueName: \"kubernetes.io/projected/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf-kube-api-access-zjrmf\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:59:22.014522 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.014485 2585 generic.go:356] "Generic (PLEG): container finished" podID="0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" containerID="b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854" exitCode=0 Jul 18 11:59:22.015032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.014533 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" Jul 18 11:59:22.015032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.014554 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" event={"ID":"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf","Type":"ContainerDied","Data":"b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854"} Jul 18 11:59:22.015032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.014598 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg" event={"ID":"0e043d14-df0f-4f40-9ac5-69c2e93cfcbf","Type":"ContainerDied","Data":"a9ab30bcbdb3f96a241d0fcc6756cdcc774d98fc2778d24db5c7e20667a099c2"} Jul 18 11:59:22.015032 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.014621 2585 scope.go:117] "RemoveContainer" containerID="b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854" Jul 18 11:59:22.023364 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.023337 2585 scope.go:117] "RemoveContainer" containerID="b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854" Jul 18 11:59:22.023646 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:59:22.023624 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854\": container with ID starting with b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854 not found: ID does not exist" containerID="b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854" Jul 18 11:59:22.023710 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.023657 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854"} err="failed to get container status \"b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854\": rpc error: code = NotFound desc = could not find container \"b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854\": container with ID starting with b4bde55088073a03b9b72c4ad344d6fdec650620c75d93efcf6bbcad3fa15854 not found: ID does not exist" Jul 18 11:59:22.033687 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.033657 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg"] Jul 18 11:59:22.041627 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:22.040273 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f7319d-56c7869f49-tb9tg"] Jul 18 11:59:23.614740 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:23.614703 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" path="/var/lib/kubelet/pods/0e043d14-df0f-4f40-9ac5-69c2e93cfcbf/volumes" Jul 18 11:59:25.629011 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:25.628973 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:26.051674 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:26.051634 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:26.181218 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:26.181179 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:59:31.712573 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:31.712528 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:31.813724 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:31.813688 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:32.000556 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:32.000393 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:32.253102 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:32.253073 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:33.022135 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:33.022069 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:33.125508 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:33.125454 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:42.909958 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:42.909911 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l"] Jul 18 11:59:42.910469 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:42.910330 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" podUID="1620dbdb-1cc2-4f32-a2df-d4593383b2a9" containerName="maas-api" containerID="cri-o://91064e12e206ea5a90370215238e269f1b531cd28ec5d9454b2f820a69c562ee" gracePeriod=30 Jul 18 11:59:42.932263 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:42.932234 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:43.130481 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.130442 2585 generic.go:356] "Generic (PLEG): container finished" podID="1620dbdb-1cc2-4f32-a2df-d4593383b2a9" containerID="91064e12e206ea5a90370215238e269f1b531cd28ec5d9454b2f820a69c562ee" exitCode=0 Jul 18 11:59:43.130638 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.130504 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" event={"ID":"1620dbdb-1cc2-4f32-a2df-d4593383b2a9","Type":"ContainerDied","Data":"91064e12e206ea5a90370215238e269f1b531cd28ec5d9454b2f820a69c562ee"} Jul 18 11:59:43.165728 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.165659 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:59:43.214270 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.214237 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh4k6\" (UniqueName: \"kubernetes.io/projected/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-kube-api-access-qh4k6\") pod \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " Jul 18 11:59:43.214445 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.214318 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-openshift-service-ca\") pod \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " Jul 18 11:59:43.214445 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.214405 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-maas-api-tls\") pod \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\" (UID: \"1620dbdb-1cc2-4f32-a2df-d4593383b2a9\") " Jul 18 11:59:43.214673 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.214649 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1620dbdb-1cc2-4f32-a2df-d4593383b2a9" (UID: "1620dbdb-1cc2-4f32-a2df-d4593383b2a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:59:43.216421 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.216394 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1620dbdb-1cc2-4f32-a2df-d4593383b2a9" (UID: "1620dbdb-1cc2-4f32-a2df-d4593383b2a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:59:43.216514 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.216491 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-kube-api-access-qh4k6" (OuterVolumeSpecName: "kube-api-access-qh4k6") pod "1620dbdb-1cc2-4f32-a2df-d4593383b2a9" (UID: "1620dbdb-1cc2-4f32-a2df-d4593383b2a9"). InnerVolumeSpecName "kube-api-access-qh4k6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:59:43.316092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.316054 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:59:43.316092 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.316092 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh4k6\" (UniqueName: \"kubernetes.io/projected/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-kube-api-access-qh4k6\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:59:43.316331 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:43.316104 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1620dbdb-1cc2-4f32-a2df-d4593383b2a9-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 11:59:44.137261 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:44.137224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" event={"ID":"1620dbdb-1cc2-4f32-a2df-d4593383b2a9","Type":"ContainerDied","Data":"e115eed01873097009ee93439ea417cedabd4814f7976916ae1e1e483f4155e7"} Jul 18 11:59:44.137261 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:44.137258 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l" Jul 18 11:59:44.137754 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:44.137287 2585 scope.go:117] "RemoveContainer" containerID="91064e12e206ea5a90370215238e269f1b531cd28ec5d9454b2f820a69c562ee" Jul 18 11:59:44.157282 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:44.157243 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l"] Jul 18 11:59:44.163536 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:44.163498 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3a9e62-596d5b9b66-bw46l"] Jul 18 11:59:45.578021 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:45.577977 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 11:59:45.616349 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:45.616312 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1620dbdb-1cc2-4f32-a2df-d4593383b2a9" path="/var/lib/kubelet/pods/1620dbdb-1cc2-4f32-a2df-d4593383b2a9/volumes" Jul 18 11:59:47.445731 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:47.445695 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:47.845179 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:47.845134 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:53.580692 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:53.580657 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:53.681327 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:53.681292 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:53.854526 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:53.854440 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:54.086004 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:54.085970 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:54.458157 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:54.458122 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:54.792581 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:54.792544 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:55.009587 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:55.009549 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:58.218970 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.218929 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:58.320319 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.320279 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt"] Jul 18 11:59:58.320668 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.320656 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" containerName="maas-api" Jul 18 11:59:58.320717 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.320670 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" containerName="maas-api" Jul 18 11:59:58.320717 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.320683 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1620dbdb-1cc2-4f32-a2df-d4593383b2a9" containerName="maas-api" Jul 18 11:59:58.320717 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.320689 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1620dbdb-1cc2-4f32-a2df-d4593383b2a9" containerName="maas-api" Jul 18 11:59:58.320810 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.320741 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1620dbdb-1cc2-4f32-a2df-d4593383b2a9" containerName="maas-api" Jul 18 11:59:58.320810 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.320749 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e043d14-df0f-4f40-9ac5-69c2e93cfcbf" containerName="maas-api" Jul 18 11:59:58.323704 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.323670 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:58.325221 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.325201 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.328131 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.328110 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e3c095-serving-cert\"" Jul 18 11:59:58.328232 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.328146 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 11:59:58.329309 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.329293 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 11:59:58.329362 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.329312 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 11:59:58.336644 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.336617 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt"] Jul 18 11:59:58.408957 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.408823 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:58.454592 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.454553 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e829b1-7a86-4500-82fa-5cd85641de0b-openshift-service-ca\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.454785 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.454679 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj6rn\" (UniqueName: \"kubernetes.io/projected/64e829b1-7a86-4500-82fa-5cd85641de0b-kube-api-access-dj6rn\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.454785 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.454724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.555501 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.555455 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.555730 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.555517 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e829b1-7a86-4500-82fa-5cd85641de0b-openshift-service-ca\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.555730 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.555610 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj6rn\" (UniqueName: \"kubernetes.io/projected/64e829b1-7a86-4500-82fa-5cd85641de0b-kube-api-access-dj6rn\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.555730 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:59:58.555609 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-serving-cert: secret "maas-api-e2e-api-a-e3c095-serving-cert" not found Jul 18 11:59:58.555730 ip-10-0-142-41 kubenswrapper[2585]: E0718 11:59:58.555715 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls podName:64e829b1-7a86-4500-82fa-5cd85641de0b nodeName:}" failed. No retries permitted until 2026-07-18 11:59:59.055691263 +0000 UTC m=+2393.957455379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls") pod "maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" (UID: "64e829b1-7a86-4500-82fa-5cd85641de0b") : secret "maas-api-e2e-api-a-e3c095-serving-cert" not found Jul 18 11:59:58.556262 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.556243 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e829b1-7a86-4500-82fa-5cd85641de0b-openshift-service-ca\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.569992 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.569960 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj6rn\" (UniqueName: \"kubernetes.io/projected/64e829b1-7a86-4500-82fa-5cd85641de0b-kube-api-access-dj6rn\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:58.669343 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:58.669306 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:59.002898 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:59.002861 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:59.060850 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:59.060808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:59.063424 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:59.063402 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls\") pod \"maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:59.105636 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:59.105595 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 11:59:59.237042 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:59.236993 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 11:59:59.382490 ip-10-0-142-41 kubenswrapper[2585]: I0718 11:59:59.382461 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt"] Jul 18 11:59:59.384222 ip-10-0-142-41 kubenswrapper[2585]: W0718 11:59:59.384190 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64e829b1_7a86_4500_82fa_5cd85641de0b.slice/crio-e1f05aab58dcf1db6cb91fb8879cd657a5c18208aa288d09b99e4b65900ae966 WatchSource:0}: Error finding container e1f05aab58dcf1db6cb91fb8879cd657a5c18208aa288d09b99e4b65900ae966: Status 404 returned error can't find the container with id e1f05aab58dcf1db6cb91fb8879cd657a5c18208aa288d09b99e4b65900ae966 Jul 18 12:00:00.233105 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:00.233001 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" event={"ID":"64e829b1-7a86-4500-82fa-5cd85641de0b","Type":"ContainerStarted","Data":"166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4"} Jul 18 12:00:00.233105 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:00.233076 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 12:00:00.233105 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:00.233101 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" event={"ID":"64e829b1-7a86-4500-82fa-5cd85641de0b","Type":"ContainerStarted","Data":"e1f05aab58dcf1db6cb91fb8879cd657a5c18208aa288d09b99e4b65900ae966"} Jul 18 12:00:00.251290 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:00.251238 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" podStartSLOduration=1.86348021 podStartE2EDuration="2.251219429s" totalImagesPullingTime="387.739219ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:00:00.249986772 +0000 UTC m=+2395.151750910" watchObservedRunningTime="2026-07-18 12:00:00.251219429 +0000 UTC m=+2395.152983566" Jul 18 12:00:02.756063 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:02.756026 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:03.261539 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:03.261487 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:03.583949 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:03.583809 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:03.881617 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:03.881523 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:00:04.243291 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:04.243166 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:06.227978 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.227823 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:06.244472 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.244442 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 12:00:06.333066 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.333033 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:06.338485 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.338448 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q"] Jul 18 12:00:06.344032 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.343714 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.346997 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.346970 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1dc2c9-serving-cert\"" Jul 18 12:00:06.366761 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.366722 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q"] Jul 18 12:00:06.432677 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.432634 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-openshift-service-ca\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.432875 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.432701 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbzs4\" (UniqueName: \"kubernetes.io/projected/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-kube-api-access-sbzs4\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.432875 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.432723 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-maas-api-tls\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.496512 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.496424 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:06.533781 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.533739 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-openshift-service-ca\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.533985 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.533804 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbzs4\" (UniqueName: \"kubernetes.io/projected/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-kube-api-access-sbzs4\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.533985 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.533824 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-maas-api-tls\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.534441 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.534417 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-openshift-service-ca\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.536404 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.536379 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-maas-api-tls\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.542965 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.542934 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbzs4\" (UniqueName: \"kubernetes.io/projected/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-kube-api-access-sbzs4\") pod \"maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.596858 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.596826 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:06.656800 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.656761 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:06.802635 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.802602 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q"] Jul 18 12:00:06.804706 ip-10-0-142-41 kubenswrapper[2585]: W0718 12:00:06.804669 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40af0b7_f701_4a74_bcc3_d24eec1a17c9.slice/crio-485b0b4194d32419c35168fb0aa2d3c7250dbf1458257a67b9c9b1aab5b9c402 WatchSource:0}: Error finding container 485b0b4194d32419c35168fb0aa2d3c7250dbf1458257a67b9c9b1aab5b9c402: Status 404 returned error can't find the container with id 485b0b4194d32419c35168fb0aa2d3c7250dbf1458257a67b9c9b1aab5b9c402 Jul 18 12:00:06.922769 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:06.922731 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:07.024461 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:07.024401 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:07.270659 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:07.270624 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" event={"ID":"a40af0b7-f701-4a74-bcc3-d24eec1a17c9","Type":"ContainerStarted","Data":"485b0b4194d32419c35168fb0aa2d3c7250dbf1458257a67b9c9b1aab5b9c402"} Jul 18 12:00:08.277962 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:08.277918 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" event={"ID":"a40af0b7-f701-4a74-bcc3-d24eec1a17c9","Type":"ContainerStarted","Data":"a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958"} Jul 18 12:00:08.278404 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:08.278101 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:08.298707 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:08.298646 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" podStartSLOduration=1.809158351 podStartE2EDuration="2.298628748s" totalImagesPullingTime="489.470397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:00:08.295640079 +0000 UTC m=+2403.197404270" watchObservedRunningTime="2026-07-18 12:00:08.298628748 +0000 UTC m=+2403.200392887" Jul 18 12:00:10.740538 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:10.740499 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:14.289523 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:14.289488 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:00:19.275042 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:19.275003 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:00:19.452299 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:19.452265 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:19.552800 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:19.552716 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:24.016023 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:24.015958 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:24.118065 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:24.118026 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:27.885568 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:27.885488 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:27.986457 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:27.986426 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:29.636313 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:29.636274 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:29.733167 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:29.733128 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:34.855342 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:34.855299 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt"] Jul 18 12:00:34.855824 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:34.855572 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" podUID="64e829b1-7a86-4500-82fa-5cd85641de0b" containerName="maas-api" containerID="cri-o://166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4" gracePeriod=30 Jul 18 12:00:34.900546 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:34.900506 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:35.109714 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.109643 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 12:00:35.184382 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.184346 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj6rn\" (UniqueName: \"kubernetes.io/projected/64e829b1-7a86-4500-82fa-5cd85641de0b-kube-api-access-dj6rn\") pod \"64e829b1-7a86-4500-82fa-5cd85641de0b\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " Jul 18 12:00:35.184382 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.184391 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls\") pod \"64e829b1-7a86-4500-82fa-5cd85641de0b\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " Jul 18 12:00:35.184620 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.184441 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e829b1-7a86-4500-82fa-5cd85641de0b-openshift-service-ca\") pod \"64e829b1-7a86-4500-82fa-5cd85641de0b\" (UID: \"64e829b1-7a86-4500-82fa-5cd85641de0b\") " Jul 18 12:00:35.184918 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.184860 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64e829b1-7a86-4500-82fa-5cd85641de0b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64e829b1-7a86-4500-82fa-5cd85641de0b" (UID: "64e829b1-7a86-4500-82fa-5cd85641de0b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:00:35.186600 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.186574 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64e829b1-7a86-4500-82fa-5cd85641de0b" (UID: "64e829b1-7a86-4500-82fa-5cd85641de0b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 12:00:35.186693 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.186639 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64e829b1-7a86-4500-82fa-5cd85641de0b-kube-api-access-dj6rn" (OuterVolumeSpecName: "kube-api-access-dj6rn") pod "64e829b1-7a86-4500-82fa-5cd85641de0b" (UID: "64e829b1-7a86-4500-82fa-5cd85641de0b"). InnerVolumeSpecName "kube-api-access-dj6rn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:00:35.285582 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.285547 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj6rn\" (UniqueName: \"kubernetes.io/projected/64e829b1-7a86-4500-82fa-5cd85641de0b-kube-api-access-dj6rn\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:00:35.285582 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.285578 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64e829b1-7a86-4500-82fa-5cd85641de0b-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:00:35.285582 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.285587 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e829b1-7a86-4500-82fa-5cd85641de0b-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:00:35.423250 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.423154 2585 generic.go:356] "Generic (PLEG): container finished" podID="64e829b1-7a86-4500-82fa-5cd85641de0b" containerID="166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4" exitCode=0 Jul 18 12:00:35.423250 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.423217 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" Jul 18 12:00:35.423525 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.423242 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" event={"ID":"64e829b1-7a86-4500-82fa-5cd85641de0b","Type":"ContainerDied","Data":"166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4"} Jul 18 12:00:35.423525 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.423331 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt" event={"ID":"64e829b1-7a86-4500-82fa-5cd85641de0b","Type":"ContainerDied","Data":"e1f05aab58dcf1db6cb91fb8879cd657a5c18208aa288d09b99e4b65900ae966"} Jul 18 12:00:35.423525 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.423355 2585 scope.go:117] "RemoveContainer" containerID="166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4" Jul 18 12:00:35.435722 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.435697 2585 scope.go:117] "RemoveContainer" containerID="166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4" Jul 18 12:00:35.436052 ip-10-0-142-41 kubenswrapper[2585]: E0718 12:00:35.436029 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4\": container with ID starting with 166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4 not found: ID does not exist" containerID="166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4" Jul 18 12:00:35.436117 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.436062 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4"} err="failed to get container status \"166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4\": rpc error: code = NotFound desc = could not find container \"166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4\": container with ID starting with 166199bbc7a286688da2c93fc2b11b1900375188434a460bc9ad2e65a5dc15f4 not found: ID does not exist" Jul 18 12:00:35.449956 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.449917 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt"] Jul 18 12:00:35.453739 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.453710 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e3c095-686cb88d87-4cnmt"] Jul 18 12:00:35.614759 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:35.614724 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64e829b1-7a86-4500-82fa-5cd85641de0b" path="/var/lib/kubelet/pods/64e829b1-7a86-4500-82fa-5cd85641de0b/volumes" Jul 18 12:00:37.676288 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:37.676244 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:00:39.003099 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:39.003063 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:39.426156 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:39.426074 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:45.288025 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:45.287993 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:45.370503 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:45.370463 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:45.532419 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:45.532379 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:46.167407 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:46.167367 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:46.265139 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:46.265087 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:48.858480 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:48.858440 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:48.909075 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:48.909039 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:00:54.198047 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:00:54.198007 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:01.109252 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.109213 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q"] Jul 18 12:01:01.109793 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.109507 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" podUID="a40af0b7-f701-4a74-bcc3-d24eec1a17c9" containerName="maas-api" containerID="cri-o://a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958" gracePeriod=30 Jul 18 12:01:01.138275 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.138242 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:01.366683 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.366608 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:01:01.404532 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.404495 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbzs4\" (UniqueName: \"kubernetes.io/projected/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-kube-api-access-sbzs4\") pod \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " Jul 18 12:01:01.404739 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.404540 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-openshift-service-ca\") pod \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " Jul 18 12:01:01.404739 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.404559 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-maas-api-tls\") pod \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\" (UID: \"a40af0b7-f701-4a74-bcc3-d24eec1a17c9\") " Jul 18 12:01:01.405021 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.404996 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a40af0b7-f701-4a74-bcc3-d24eec1a17c9" (UID: "a40af0b7-f701-4a74-bcc3-d24eec1a17c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:01:01.406773 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.406729 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-kube-api-access-sbzs4" (OuterVolumeSpecName: "kube-api-access-sbzs4") pod "a40af0b7-f701-4a74-bcc3-d24eec1a17c9" (UID: "a40af0b7-f701-4a74-bcc3-d24eec1a17c9"). InnerVolumeSpecName "kube-api-access-sbzs4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:01:01.406872 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.406771 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a40af0b7-f701-4a74-bcc3-d24eec1a17c9" (UID: "a40af0b7-f701-4a74-bcc3-d24eec1a17c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 12:01:01.505533 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.505492 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbzs4\" (UniqueName: \"kubernetes.io/projected/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-kube-api-access-sbzs4\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:01:01.505533 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.505525 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:01:01.505533 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.505535 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a40af0b7-f701-4a74-bcc3-d24eec1a17c9-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:01:01.569343 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.569301 2585 generic.go:356] "Generic (PLEG): container finished" podID="a40af0b7-f701-4a74-bcc3-d24eec1a17c9" containerID="a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958" exitCode=0 Jul 18 12:01:01.569343 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.569340 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" Jul 18 12:01:01.569552 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.569380 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" event={"ID":"a40af0b7-f701-4a74-bcc3-d24eec1a17c9","Type":"ContainerDied","Data":"a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958"} Jul 18 12:01:01.569552 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.569435 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q" event={"ID":"a40af0b7-f701-4a74-bcc3-d24eec1a17c9","Type":"ContainerDied","Data":"485b0b4194d32419c35168fb0aa2d3c7250dbf1458257a67b9c9b1aab5b9c402"} Jul 18 12:01:01.569552 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.569460 2585 scope.go:117] "RemoveContainer" containerID="a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958" Jul 18 12:01:01.579258 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.579236 2585 scope.go:117] "RemoveContainer" containerID="a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958" Jul 18 12:01:01.585927 ip-10-0-142-41 kubenswrapper[2585]: E0718 12:01:01.583942 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958\": container with ID starting with a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958 not found: ID does not exist" containerID="a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958" Jul 18 12:01:01.585927 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.584009 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958"} err="failed to get container status \"a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958\": rpc error: code = NotFound desc = could not find container \"a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958\": container with ID starting with a9a1819623d6bb9f96fddf85a61d0b6af8cd6fe056b03d2d10b7d4da682fc958 not found: ID does not exist" Jul 18 12:01:01.592342 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.592308 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q"] Jul 18 12:01:01.596350 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.596318 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dc2c9-77f5d547df-7mt9q"] Jul 18 12:01:01.614719 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:01.614682 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a40af0b7-f701-4a74-bcc3-d24eec1a17c9" path="/var/lib/kubelet/pods/a40af0b7-f701-4a74-bcc3-d24eec1a17c9/volumes" Jul 18 12:01:05.672373 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:05.672333 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:01:05.727674 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:05.727609 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:06.132627 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:06.132589 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:12.042484 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:12.042451 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:12.099993 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:12.099958 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:12.320701 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:12.320580 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:12.564601 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:12.564566 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:12.905236 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:12.905191 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:13.251790 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:13.251702 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:13.600164 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:13.600130 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:16.708137 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.708097 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:16.808529 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.808495 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:16.879727 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.879681 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn"] Jul 18 12:01:16.880142 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.880125 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40af0b7-f701-4a74-bcc3-d24eec1a17c9" containerName="maas-api" Jul 18 12:01:16.880231 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.880143 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40af0b7-f701-4a74-bcc3-d24eec1a17c9" containerName="maas-api" Jul 18 12:01:16.880231 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.880163 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64e829b1-7a86-4500-82fa-5cd85641de0b" containerName="maas-api" Jul 18 12:01:16.880231 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.880168 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="64e829b1-7a86-4500-82fa-5cd85641de0b" containerName="maas-api" Jul 18 12:01:16.880231 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.880226 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a40af0b7-f701-4a74-bcc3-d24eec1a17c9" containerName="maas-api" Jul 18 12:01:16.880375 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.880238 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="64e829b1-7a86-4500-82fa-5cd85641de0b" containerName="maas-api" Jul 18 12:01:16.883206 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.883174 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:16.885780 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.885745 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-399bd8-serving-cert\"" Jul 18 12:01:16.885953 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.885871 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 12:01:16.885953 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.885910 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 12:01:16.887076 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.887052 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qwdvw\"" Jul 18 12:01:16.895396 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.895358 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn"] Jul 18 12:01:16.917132 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:16.917095 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:17.043162 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.043123 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97555a9b-dc80-4350-bf63-6a65325ba0dc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.043162 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.043164 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.043420 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.043224 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7hvs\" (UniqueName: \"kubernetes.io/projected/97555a9b-dc80-4350-bf63-6a65325ba0dc-kube-api-access-w7hvs\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.144045 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.143998 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97555a9b-dc80-4350-bf63-6a65325ba0dc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.144045 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.144047 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.144304 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.144085 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7hvs\" (UniqueName: \"kubernetes.io/projected/97555a9b-dc80-4350-bf63-6a65325ba0dc-kube-api-access-w7hvs\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.144304 ip-10-0-142-41 kubenswrapper[2585]: E0718 12:01:17.144243 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-serving-cert: secret "maas-api-e2e-shared-a-399bd8-serving-cert" not found Jul 18 12:01:17.144413 ip-10-0-142-41 kubenswrapper[2585]: E0718 12:01:17.144335 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls podName:97555a9b-dc80-4350-bf63-6a65325ba0dc nodeName:}" failed. No retries permitted until 2026-07-18 12:01:17.644311835 +0000 UTC m=+2472.546075950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls") pod "maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" (UID: "97555a9b-dc80-4350-bf63-6a65325ba0dc") : secret "maas-api-e2e-shared-a-399bd8-serving-cert" not found Jul 18 12:01:17.144713 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.144694 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97555a9b-dc80-4350-bf63-6a65325ba0dc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.157016 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.156982 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7hvs\" (UniqueName: \"kubernetes.io/projected/97555a9b-dc80-4350-bf63-6a65325ba0dc-kube-api-access-w7hvs\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.318048 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.317952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:17.448248 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.448210 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:17.549048 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.549016 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:17.649198 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.649101 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.651861 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.651833 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls\") pod \"maas-api-e2e-shared-a-399bd8-66b7849988-78mtn\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.796331 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.796291 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:17.942360 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:17.942287 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn"] Jul 18 12:01:17.944755 ip-10-0-142-41 kubenswrapper[2585]: W0718 12:01:17.944724 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97555a9b_dc80_4350_bf63_6a65325ba0dc.slice/crio-6df70182e2bd990a48c91a3cc51df4a87bd32e6abb500006b189f9ee8125563e WatchSource:0}: Error finding container 6df70182e2bd990a48c91a3cc51df4a87bd32e6abb500006b189f9ee8125563e: Status 404 returned error can't find the container with id 6df70182e2bd990a48c91a3cc51df4a87bd32e6abb500006b189f9ee8125563e Jul 18 12:01:18.661579 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:18.661474 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" event={"ID":"97555a9b-dc80-4350-bf63-6a65325ba0dc","Type":"ContainerStarted","Data":"4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b"} Jul 18 12:01:18.661579 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:18.661533 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" event={"ID":"97555a9b-dc80-4350-bf63-6a65325ba0dc","Type":"ContainerStarted","Data":"6df70182e2bd990a48c91a3cc51df4a87bd32e6abb500006b189f9ee8125563e"} Jul 18 12:01:18.661579 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:18.661568 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:18.688085 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:18.688025 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" podStartSLOduration=2.318414869 podStartE2EDuration="2.688004832s" totalImagesPullingTime="369.589963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:01:18.686249126 +0000 UTC m=+2473.588013263" watchObservedRunningTime="2026-07-18 12:01:18.688004832 +0000 UTC m=+2473.589768971" Jul 18 12:01:21.172590 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:21.172555 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:21.514053 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:21.514017 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:21.845673 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:21.845536 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:22.091720 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:22.091669 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:24.673205 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.673174 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:01:24.678817 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.678784 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:24.775220 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.775172 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:24.884337 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.884300 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td"] Jul 18 12:01:24.887981 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.887954 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:24.890534 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.890512 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-276d7e-serving-cert\"" Jul 18 12:01:24.899713 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.899637 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td"] Jul 18 12:01:24.944744 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:24.944653 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:25.016456 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.016414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2gwv\" (UniqueName: \"kubernetes.io/projected/d30faee4-a633-48ac-890d-80d10ba10625-kube-api-access-j2gwv\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.016721 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.016478 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30faee4-a633-48ac-890d-80d10ba10625-openshift-service-ca\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.016721 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.016595 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30faee4-a633-48ac-890d-80d10ba10625-maas-api-tls\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.043768 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.043736 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:25.117737 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.117694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2gwv\" (UniqueName: \"kubernetes.io/projected/d30faee4-a633-48ac-890d-80d10ba10625-kube-api-access-j2gwv\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.117938 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.117753 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30faee4-a633-48ac-890d-80d10ba10625-openshift-service-ca\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.117938 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.117783 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30faee4-a633-48ac-890d-80d10ba10625-maas-api-tls\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.118458 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.118435 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30faee4-a633-48ac-890d-80d10ba10625-openshift-service-ca\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.120224 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.120203 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30faee4-a633-48ac-890d-80d10ba10625-maas-api-tls\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.126685 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.126655 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2gwv\" (UniqueName: \"kubernetes.io/projected/d30faee4-a633-48ac-890d-80d10ba10625-kube-api-access-j2gwv\") pod \"maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.200300 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.200202 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:25.345018 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.344915 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td"] Jul 18 12:01:25.348082 ip-10-0-142-41 kubenswrapper[2585]: W0718 12:01:25.348047 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30faee4_a633_48ac_890d_80d10ba10625.slice/crio-423fe8b5ef1e591637f30e4d94c010ce03196c75137170f33471a7ecad55074e WatchSource:0}: Error finding container 423fe8b5ef1e591637f30e4d94c010ce03196c75137170f33471a7ecad55074e: Status 404 returned error can't find the container with id 423fe8b5ef1e591637f30e4d94c010ce03196c75137170f33471a7ecad55074e Jul 18 12:01:25.477150 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.477063 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:01:25.565433 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.565400 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:25.666963 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.666927 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:25.703134 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:25.703093 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" event={"ID":"d30faee4-a633-48ac-890d-80d10ba10625","Type":"ContainerStarted","Data":"423fe8b5ef1e591637f30e4d94c010ce03196c75137170f33471a7ecad55074e"} Jul 18 12:01:26.709749 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:26.709710 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" event={"ID":"d30faee4-a633-48ac-890d-80d10ba10625","Type":"ContainerStarted","Data":"f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a"} Jul 18 12:01:26.710223 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:26.709798 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:26.728943 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:26.728850 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" podStartSLOduration=2.313612794 podStartE2EDuration="2.728828683s" totalImagesPullingTime="415.215889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:01:26.727338302 +0000 UTC m=+2481.629102441" watchObservedRunningTime="2026-07-18 12:01:26.728828683 +0000 UTC m=+2481.630592824" Jul 18 12:01:29.181381 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:29.181341 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:32.722037 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:32.722007 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:01:37.153915 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:37.153861 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:39.241175 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:39.241117 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:41.081700 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:41.081657 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:01:46.609472 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:46.609430 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:01:46.708716 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:01:46.708685 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:02:20.163496 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:02:20.163457 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:02:23.773251 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:02:23.773210 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:02:37.573112 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:02:37.573069 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:02:53.281350 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:02:53.281311 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:03:07.774541 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:03:07.774499 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:03:24.973234 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:03:24.973194 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:03:43.469936 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:03:43.469818 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:03:59.983982 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:03:59.983932 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:04:15.656004 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:04:15.655960 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:04:16.457831 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:04:16.457784 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:04:18.781226 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:04:18.781184 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:04:32.076050 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:04:32.076009 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:04:32.235334 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:04:32.235294 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:04:36.119200 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:04:36.119149 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:04:41.685734 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:04:41.685698 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:05:15.757530 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:15.757483 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:19.777343 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:19.777309 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:05:22.934416 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:22.934380 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:24.785903 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:24.785860 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:24.886629 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:24.886574 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:34.075328 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:34.075292 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:05:50.924456 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:50.924420 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:50.985973 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:50.985930 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:51.046493 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:51.046451 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:51.824583 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:51.824551 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:05:57.076143 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:05:57.076104 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:06:03.666955 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:03.666918 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:06:06.978552 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:06.978514 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:06:31.964417 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:31.964337 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:06:35.481394 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:35.481343 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:06:39.336808 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:39.336761 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:06:40.775604 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:40.775560 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:06:47.434835 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:47.434796 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:06:47.538838 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:47.538771 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:06:56.283065 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:06:56.283023 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:07:03.995097 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:03.995060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:11.176609 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:11.176571 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:07:18.180901 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:18.180837 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:19.136413 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:19.136378 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:26.714912 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:26.714832 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:32.176755 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:32.176715 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:07:38.424396 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:38.424330 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:38.522234 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:38.522180 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:48.330423 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.330384 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn"] Jul 18 12:07:48.330952 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.330617 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" podUID="97555a9b-dc80-4350-bf63-6a65325ba0dc" containerName="maas-api" containerID="cri-o://4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b" gracePeriod=30 Jul 18 12:07:48.352515 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.352486 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:48.591296 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.591207 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:07:48.682950 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.682867 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97555a9b-dc80-4350-bf63-6a65325ba0dc-openshift-service-ca\") pod \"97555a9b-dc80-4350-bf63-6a65325ba0dc\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " Jul 18 12:07:48.682950 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.682953 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls\") pod \"97555a9b-dc80-4350-bf63-6a65325ba0dc\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " Jul 18 12:07:48.683248 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.682985 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7hvs\" (UniqueName: \"kubernetes.io/projected/97555a9b-dc80-4350-bf63-6a65325ba0dc-kube-api-access-w7hvs\") pod \"97555a9b-dc80-4350-bf63-6a65325ba0dc\" (UID: \"97555a9b-dc80-4350-bf63-6a65325ba0dc\") " Jul 18 12:07:48.683314 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.683266 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97555a9b-dc80-4350-bf63-6a65325ba0dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97555a9b-dc80-4350-bf63-6a65325ba0dc" (UID: "97555a9b-dc80-4350-bf63-6a65325ba0dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:07:48.685312 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.685277 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97555a9b-dc80-4350-bf63-6a65325ba0dc-kube-api-access-w7hvs" (OuterVolumeSpecName: "kube-api-access-w7hvs") pod "97555a9b-dc80-4350-bf63-6a65325ba0dc" (UID: "97555a9b-dc80-4350-bf63-6a65325ba0dc"). InnerVolumeSpecName "kube-api-access-w7hvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:07:48.685415 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.685307 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97555a9b-dc80-4350-bf63-6a65325ba0dc" (UID: "97555a9b-dc80-4350-bf63-6a65325ba0dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 12:07:48.773339 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.773302 2585 generic.go:356] "Generic (PLEG): container finished" podID="97555a9b-dc80-4350-bf63-6a65325ba0dc" containerID="4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b" exitCode=0 Jul 18 12:07:48.773510 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.773354 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" Jul 18 12:07:48.773510 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.773389 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" event={"ID":"97555a9b-dc80-4350-bf63-6a65325ba0dc","Type":"ContainerDied","Data":"4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b"} Jul 18 12:07:48.773510 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.773454 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn" event={"ID":"97555a9b-dc80-4350-bf63-6a65325ba0dc","Type":"ContainerDied","Data":"6df70182e2bd990a48c91a3cc51df4a87bd32e6abb500006b189f9ee8125563e"} Jul 18 12:07:48.773510 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.773485 2585 scope.go:117] "RemoveContainer" containerID="4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b" Jul 18 12:07:48.783380 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.783354 2585 scope.go:117] "RemoveContainer" containerID="4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b" Jul 18 12:07:48.783740 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.783718 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97555a9b-dc80-4350-bf63-6a65325ba0dc-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:07:48.783791 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.783743 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97555a9b-dc80-4350-bf63-6a65325ba0dc-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:07:48.783791 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.783752 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7hvs\" (UniqueName: \"kubernetes.io/projected/97555a9b-dc80-4350-bf63-6a65325ba0dc-kube-api-access-w7hvs\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:07:48.783791 ip-10-0-142-41 kubenswrapper[2585]: E0718 12:07:48.783767 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b\": container with ID starting with 4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b not found: ID does not exist" containerID="4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b" Jul 18 12:07:48.783922 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.783798 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b"} err="failed to get container status \"4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b\": rpc error: code = NotFound desc = could not find container \"4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b\": container with ID starting with 4e0fc496d02a55420cd31eeff321b6bae920fe7425a9daeebe057f122eb7a87b not found: ID does not exist" Jul 18 12:07:48.798695 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.798653 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn"] Jul 18 12:07:48.802630 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:48.802597 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-399bd8-66b7849988-78mtn"] Jul 18 12:07:49.614384 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:49.614344 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97555a9b-dc80-4350-bf63-6a65325ba0dc" path="/var/lib/kubelet/pods/97555a9b-dc80-4350-bf63-6a65325ba0dc/volumes" Jul 18 12:07:51.378665 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:51.378629 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:07:52.987601 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:52.987567 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:53.418486 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:53.418448 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:59.323522 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:59.323441 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:59.385816 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:59.385783 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:59.668951 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:59.668844 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:07:59.864746 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:07:59.864703 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:00.509849 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:00.509818 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:00.609863 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:00.609828 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:10.391613 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.391570 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td"] Jul 18 12:08:10.392082 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.391841 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" podUID="d30faee4-a633-48ac-890d-80d10ba10625" containerName="maas-api" containerID="cri-o://f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a" gracePeriod=30 Jul 18 12:08:10.414547 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.414518 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:10.648554 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.648479 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:08:10.789064 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.789023 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30faee4-a633-48ac-890d-80d10ba10625-maas-api-tls\") pod \"d30faee4-a633-48ac-890d-80d10ba10625\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " Jul 18 12:08:10.789276 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.789091 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30faee4-a633-48ac-890d-80d10ba10625-openshift-service-ca\") pod \"d30faee4-a633-48ac-890d-80d10ba10625\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " Jul 18 12:08:10.789276 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.789163 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2gwv\" (UniqueName: \"kubernetes.io/projected/d30faee4-a633-48ac-890d-80d10ba10625-kube-api-access-j2gwv\") pod \"d30faee4-a633-48ac-890d-80d10ba10625\" (UID: \"d30faee4-a633-48ac-890d-80d10ba10625\") " Jul 18 12:08:10.789553 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.789511 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d30faee4-a633-48ac-890d-80d10ba10625-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d30faee4-a633-48ac-890d-80d10ba10625" (UID: "d30faee4-a633-48ac-890d-80d10ba10625"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:08:10.791263 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.791243 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d30faee4-a633-48ac-890d-80d10ba10625-kube-api-access-j2gwv" (OuterVolumeSpecName: "kube-api-access-j2gwv") pod "d30faee4-a633-48ac-890d-80d10ba10625" (UID: "d30faee4-a633-48ac-890d-80d10ba10625"). InnerVolumeSpecName "kube-api-access-j2gwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:08:10.791341 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.791328 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d30faee4-a633-48ac-890d-80d10ba10625-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d30faee4-a633-48ac-890d-80d10ba10625" (UID: "d30faee4-a633-48ac-890d-80d10ba10625"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 12:08:10.889943 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.889908 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30faee4-a633-48ac-890d-80d10ba10625-maas-api-tls\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:08:10.889943 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.889937 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30faee4-a633-48ac-890d-80d10ba10625-openshift-service-ca\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:08:10.889943 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.889950 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2gwv\" (UniqueName: \"kubernetes.io/projected/d30faee4-a633-48ac-890d-80d10ba10625-kube-api-access-j2gwv\") on node \"ip-10-0-142-41.ec2.internal\" DevicePath \"\"" Jul 18 12:08:10.894125 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.894091 2585 generic.go:356] "Generic (PLEG): container finished" podID="d30faee4-a633-48ac-890d-80d10ba10625" containerID="f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a" exitCode=0 Jul 18 12:08:10.894237 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.894135 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" Jul 18 12:08:10.894237 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.894171 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" event={"ID":"d30faee4-a633-48ac-890d-80d10ba10625","Type":"ContainerDied","Data":"f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a"} Jul 18 12:08:10.894237 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.894223 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td" event={"ID":"d30faee4-a633-48ac-890d-80d10ba10625","Type":"ContainerDied","Data":"423fe8b5ef1e591637f30e4d94c010ce03196c75137170f33471a7ecad55074e"} Jul 18 12:08:10.894365 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.894247 2585 scope.go:117] "RemoveContainer" containerID="f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a" Jul 18 12:08:10.903586 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.903569 2585 scope.go:117] "RemoveContainer" containerID="f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a" Jul 18 12:08:10.903845 ip-10-0-142-41 kubenswrapper[2585]: E0718 12:08:10.903824 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a\": container with ID starting with f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a not found: ID does not exist" containerID="f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a" Jul 18 12:08:10.903932 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.903857 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a"} err="failed to get container status \"f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a\": rpc error: code = NotFound desc = could not find container \"f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a\": container with ID starting with f2c8aec9ee8ef5572ac9adc8dc22df41a4620843de148a74155a463227daa94a not found: ID does not exist" Jul 18 12:08:10.916383 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.916350 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td"] Jul 18 12:08:10.919659 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:10.919629 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-276d7e-ffbd48d6f-rz9td"] Jul 18 12:08:11.614444 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:11.614411 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d30faee4-a633-48ac-890d-80d10ba10625" path="/var/lib/kubelet/pods/d30faee4-a633-48ac-890d-80d10ba10625/volumes" Jul 18 12:08:13.470754 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:13.470709 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6vwhj"] Jul 18 12:08:15.061922 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:15.061862 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:15.482950 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:15.482838 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:21.511581 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:21.511531 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:21.613282 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:21.613221 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:21.795473 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:21.795390 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:22.044485 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:22.044431 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:27.924374 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:27.924341 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kjlvj_04ef4102-2a02-4880-bcf7-0e8f31186a93/manager/0.log" Jul 18 12:08:28.531026 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:28.530997 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b8cdd6f7-gn657_d173da93-9010-42f8-9d6b-056af8fe7613/copy-manifests/0.log" Jul 18 12:08:28.543958 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:28.543925 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b8cdd6f7-gn657_d173da93-9010-42f8-9d6b-056af8fe7613/manager/0.log" Jul 18 12:08:30.265858 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.265823 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz_c5e28c4d-981c-479c-9dc2-3a84f8769386/util/0.log" Jul 18 12:08:30.272347 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.272326 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz_c5e28c4d-981c-479c-9dc2-3a84f8769386/pull/0.log" Jul 18 12:08:30.278365 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.278346 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz_c5e28c4d-981c-479c-9dc2-3a84f8769386/extract/0.log" Jul 18 12:08:30.396615 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.396586 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb_07e097f8-f1b8-4c1d-a86a-0728eea9b93d/util/0.log" Jul 18 12:08:30.403191 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.403156 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb_07e097f8-f1b8-4c1d-a86a-0728eea9b93d/pull/0.log" Jul 18 12:08:30.409493 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.409457 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb_07e097f8-f1b8-4c1d-a86a-0728eea9b93d/extract/0.log" Jul 18 12:08:30.527977 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.527900 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8_a7bff23c-5278-43fe-bb17-c7ce37bc02d8/extract/0.log" Jul 18 12:08:30.534562 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.534535 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8_a7bff23c-5278-43fe-bb17-c7ce37bc02d8/util/0.log" Jul 18 12:08:30.541364 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.541333 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8_a7bff23c-5278-43fe-bb17-c7ce37bc02d8/pull/0.log" Jul 18 12:08:30.658004 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.657975 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd_8eee5fd2-40ff-4d9e-8701-83005930c682/util/0.log" Jul 18 12:08:30.664462 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.664420 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd_8eee5fd2-40ff-4d9e-8701-83005930c682/pull/0.log" Jul 18 12:08:30.670690 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:30.670663 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd_8eee5fd2-40ff-4d9e-8701-83005930c682/extract/0.log" Jul 18 12:08:31.495968 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:31.495938 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6vwhj_1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151/limitador/0.log" Jul 18 12:08:32.092801 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:32.092768 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7t8qd_bd5801a2-c674-4f9f-9122-74ec9f6eb6b0/discovery/0.log" Jul 18 12:08:33.367317 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:33.367289 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2m8x7_f99d2e80-b92f-4963-885f-2d6608964bb7/main/0.log" Jul 18 12:08:33.590857 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:33.590827 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-bt2pw_195453e1-3339-4d04-b715-3fa59cc94932/main/0.log" Jul 18 12:08:39.780180 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:39.780142 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:39.942459 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:39.942425 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" limit=1024 Jul 18 12:08:42.677780 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:42.677747 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8jxp7_d1109b33-d650-4822-b3ab-08d1cd14d5a4/global-pull-secret-syncer/0.log" Jul 18 12:08:42.764047 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:42.764011 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gx6rn_69157e6f-b546-4043-897f-e914056249c5/konnectivity-agent/0.log" Jul 18 12:08:42.896828 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:42.896797 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-41.ec2.internal_5cb379f3a8937535961aac401e8f7558/haproxy/0.log" Jul 18 12:08:47.074143 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.074093 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz_c5e28c4d-981c-479c-9dc2-3a84f8769386/extract/0.log" Jul 18 12:08:47.097941 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.097909 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz_c5e28c4d-981c-479c-9dc2-3a84f8769386/util/0.log" Jul 18 12:08:47.121009 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.120980 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hk8tz_c5e28c4d-981c-479c-9dc2-3a84f8769386/pull/0.log" Jul 18 12:08:47.151086 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.151058 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb_07e097f8-f1b8-4c1d-a86a-0728eea9b93d/extract/0.log" Jul 18 12:08:47.178591 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.178556 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb_07e097f8-f1b8-4c1d-a86a-0728eea9b93d/util/0.log" Jul 18 12:08:47.202316 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.202284 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kjjmb_07e097f8-f1b8-4c1d-a86a-0728eea9b93d/pull/0.log" Jul 18 12:08:47.238410 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.238375 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8_a7bff23c-5278-43fe-bb17-c7ce37bc02d8/extract/0.log" Jul 18 12:08:47.264598 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.264548 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8_a7bff23c-5278-43fe-bb17-c7ce37bc02d8/util/0.log" Jul 18 12:08:47.292341 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.292311 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73sjhz8_a7bff23c-5278-43fe-bb17-c7ce37bc02d8/pull/0.log" Jul 18 12:08:47.320496 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.320448 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd_8eee5fd2-40ff-4d9e-8701-83005930c682/extract/0.log" Jul 18 12:08:47.347175 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.347076 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd_8eee5fd2-40ff-4d9e-8701-83005930c682/util/0.log" Jul 18 12:08:47.367818 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:47.367775 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17vnqd_8eee5fd2-40ff-4d9e-8701-83005930c682/pull/0.log" Jul 18 12:08:48.009254 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:48.009209 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6vwhj_1a1cd3f8-425e-4eeb-9b54-9c5a3ed2a151/limitador/0.log" Jul 18 12:08:49.872523 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:49.872490 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-n5l4t_65929cc1-658b-4d72-81b9-73f85a46a9eb/kube-state-metrics/0.log" Jul 18 12:08:49.889448 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:49.889418 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-n5l4t_65929cc1-658b-4d72-81b9-73f85a46a9eb/kube-rbac-proxy-main/0.log" Jul 18 12:08:49.910172 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:49.910143 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-n5l4t_65929cc1-658b-4d72-81b9-73f85a46a9eb/kube-rbac-proxy-self/0.log" Jul 18 12:08:49.937594 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:49.937565 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f98fc96b-v85sn_0d22b06b-8d48-4e69-9174-8428c1685dc4/metrics-server/0.log" Jul 18 12:08:49.968290 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:49.968239 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-vg826_d8c1cdb1-d0b2-49cf-a3dd-a53f041fdaa2/monitoring-plugin/0.log" Jul 18 12:08:50.158690 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.158616 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxrfk_a7154b70-e5dd-4f5a-b51a-41ad727aab16/node-exporter/0.log" Jul 18 12:08:50.178028 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.177995 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxrfk_a7154b70-e5dd-4f5a-b51a-41ad727aab16/kube-rbac-proxy/0.log" Jul 18 12:08:50.200622 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.200597 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kxrfk_a7154b70-e5dd-4f5a-b51a-41ad727aab16/init-textfile/0.log" Jul 18 12:08:50.229784 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.229754 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-z2tpm_f743c850-a5e0-4607-ba56-16bb19c1d0f8/kube-rbac-proxy-main/0.log" Jul 18 12:08:50.250859 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.250827 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-z2tpm_f743c850-a5e0-4607-ba56-16bb19c1d0f8/kube-rbac-proxy-self/0.log" Jul 18 12:08:50.273330 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.273291 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-z2tpm_f743c850-a5e0-4607-ba56-16bb19c1d0f8/openshift-state-metrics/0.log" Jul 18 12:08:50.595036 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.595004 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-tmvk5_c69bd109-a8f7-476d-ac91-375617cc0d52/prometheus-operator/0.log" Jul 18 12:08:50.613826 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.613797 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-tmvk5_c69bd109-a8f7-476d-ac91-375617cc0d52/kube-rbac-proxy/0.log" Jul 18 12:08:50.666624 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.666577 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b6fbf784-t68rd_4d310666-ad0f-44b4-a218-9e13367fd168/telemeter-client/0.log" Jul 18 12:08:50.686186 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.686155 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b6fbf784-t68rd_4d310666-ad0f-44b4-a218-9e13367fd168/reload/0.log" Jul 18 12:08:50.711589 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.711564 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b6fbf784-t68rd_4d310666-ad0f-44b4-a218-9e13367fd168/kube-rbac-proxy/0.log" Jul 18 12:08:50.747675 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.747641 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65f7c6c966-wqwxs_686b7d6c-37f5-4082-8ed4-49b0d648fa18/thanos-query/0.log" Jul 18 12:08:50.767063 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.767031 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65f7c6c966-wqwxs_686b7d6c-37f5-4082-8ed4-49b0d648fa18/kube-rbac-proxy-web/0.log" Jul 18 12:08:50.786379 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.786351 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65f7c6c966-wqwxs_686b7d6c-37f5-4082-8ed4-49b0d648fa18/kube-rbac-proxy/0.log" Jul 18 12:08:50.805418 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.805390 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65f7c6c966-wqwxs_686b7d6c-37f5-4082-8ed4-49b0d648fa18/prom-label-proxy/0.log" Jul 18 12:08:50.827085 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.827058 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65f7c6c966-wqwxs_686b7d6c-37f5-4082-8ed4-49b0d648fa18/kube-rbac-proxy-rules/0.log" Jul 18 12:08:50.847507 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:50.847427 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65f7c6c966-wqwxs_686b7d6c-37f5-4082-8ed4-49b0d648fa18/kube-rbac-proxy-metrics/0.log" Jul 18 12:08:52.956965 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.956924 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr"] Jul 18 12:08:52.957376 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.957317 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97555a9b-dc80-4350-bf63-6a65325ba0dc" containerName="maas-api" Jul 18 12:08:52.957376 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.957329 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="97555a9b-dc80-4350-bf63-6a65325ba0dc" containerName="maas-api" Jul 18 12:08:52.957376 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.957348 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d30faee4-a633-48ac-890d-80d10ba10625" containerName="maas-api" Jul 18 12:08:52.957376 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.957354 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d30faee4-a633-48ac-890d-80d10ba10625" containerName="maas-api" Jul 18 12:08:52.957513 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.957421 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="97555a9b-dc80-4350-bf63-6a65325ba0dc" containerName="maas-api" Jul 18 12:08:52.957513 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.957434 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d30faee4-a633-48ac-890d-80d10ba10625" containerName="maas-api" Jul 18 12:08:52.960620 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.960599 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:52.963633 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.963609 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9kfp5\"/\"default-dockercfg-jf7sl\"" Jul 18 12:08:52.963775 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.963672 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9kfp5\"/\"openshift-service-ca.crt\"" Jul 18 12:08:52.965154 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.965128 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9kfp5\"/\"kube-root-ca.crt\"" Jul 18 12:08:52.971312 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:52.971279 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr"] Jul 18 12:08:53.030945 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.030872 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-proc\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.031133 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.030973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6fb\" (UniqueName: \"kubernetes.io/projected/9841ac99-c5fd-43c9-be9c-0759546b5a26-kube-api-access-vt6fb\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.031133 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.031011 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-sys\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.031133 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.031029 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-podres\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.031133 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.031106 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-lib-modules\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.131741 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131699 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-sys\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.131741 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131734 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-podres\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.132030 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131770 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-lib-modules\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.132030 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131812 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-proc\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.132030 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131833 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-sys\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.132030 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131920 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-podres\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.132030 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-proc\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.132030 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131949 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6fb\" (UniqueName: \"kubernetes.io/projected/9841ac99-c5fd-43c9-be9c-0759546b5a26-kube-api-access-vt6fb\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.132030 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.131979 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9841ac99-c5fd-43c9-be9c-0759546b5a26-lib-modules\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.139755 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.139728 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6fb\" (UniqueName: \"kubernetes.io/projected/9841ac99-c5fd-43c9-be9c-0759546b5a26-kube-api-access-vt6fb\") pod \"perf-node-gather-daemonset-qjlkr\" (UID: \"9841ac99-c5fd-43c9-be9c-0759546b5a26\") " pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.256783 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.256751 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-mrw69_0e141688-e460-49f5-87a9-44361c652b0b/download-server/0.log" Jul 18 12:08:53.272800 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.272769 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:53.414342 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:53.414309 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr"] Jul 18 12:08:53.416629 ip-10-0-142-41 kubenswrapper[2585]: W0718 12:08:53.416604 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9841ac99_c5fd_43c9_be9c_0759546b5a26.slice/crio-1fd4fcc1cab8429689ee0be68b840671423efef51cf0479d0c23b520efb58c63 WatchSource:0}: Error finding container 1fd4fcc1cab8429689ee0be68b840671423efef51cf0479d0c23b520efb58c63: Status 404 returned error can't find the container with id 1fd4fcc1cab8429689ee0be68b840671423efef51cf0479d0c23b520efb58c63 Jul 18 12:08:54.120261 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:54.120224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" event={"ID":"9841ac99-c5fd-43c9-be9c-0759546b5a26","Type":"ContainerStarted","Data":"4dc3500a667793b2da7b4da8d9b2f11c78cf223d8dee4e666c17f03b0e8b190d"} Jul 18 12:08:54.120655 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:54.120276 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" event={"ID":"9841ac99-c5fd-43c9-be9c-0759546b5a26","Type":"ContainerStarted","Data":"1fd4fcc1cab8429689ee0be68b840671423efef51cf0479d0c23b520efb58c63"} Jul 18 12:08:54.120655 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:54.120368 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:08:54.140373 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:54.140307 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" podStartSLOduration=2.140288113 podStartE2EDuration="2.140288113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 12:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:08:54.138147496 +0000 UTC m=+2929.039911674" watchObservedRunningTime="2026-07-18 12:08:54.140288113 +0000 UTC m=+2929.042052253" Jul 18 12:08:55.070279 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:55.070237 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6rlm_d796e035-c9ad-4fce-a3cf-dfcc23d3992f/dns/0.log" Jul 18 12:08:55.089092 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:55.089064 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w6rlm_d796e035-c9ad-4fce-a3cf-dfcc23d3992f/kube-rbac-proxy/0.log" Jul 18 12:08:55.135570 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:55.135539 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d8qgd_6a96c874-8c04-41a0-bf56-16613a1a69c0/dns-node-resolver/0.log" Jul 18 12:08:55.729092 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:55.729062 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6tf8c_0b7c0627-1350-4093-8768-8dae93d211b9/node-ca/0.log" Jul 18 12:08:56.822589 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:56.822549 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7t8qd_bd5801a2-c674-4f9f-9122-74ec9f6eb6b0/discovery/0.log" Jul 18 12:08:57.639902 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:57.639847 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-597tz_421f75d5-537e-4d1f-9129-7713b9851728/serve-healthcheck-canary/0.log" Jul 18 12:08:58.223781 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:58.223751 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-89wsg_08c58c0e-cea2-4795-a53a-958964e776f4/kube-rbac-proxy/0.log" Jul 18 12:08:58.242357 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:58.242308 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-89wsg_08c58c0e-cea2-4795-a53a-958964e776f4/exporter/0.log" Jul 18 12:08:58.261313 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:08:58.261284 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-89wsg_08c58c0e-cea2-4795-a53a-958964e776f4/extractor/0.log" Jul 18 12:09:00.137260 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:00.137225 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9kfp5/perf-node-gather-daemonset-qjlkr" Jul 18 12:09:00.417526 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:00.417434 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kjlvj_04ef4102-2a02-4880-bcf7-0e8f31186a93/manager/0.log" Jul 18 12:09:00.916598 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:00.916563 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b8cdd6f7-gn657_d173da93-9010-42f8-9d6b-056af8fe7613/manager/0.log" Jul 18 12:09:00.932412 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:00.932382 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b8cdd6f7-gn657_d173da93-9010-42f8-9d6b-056af8fe7613/copy-manifests/0.log" Jul 18 12:09:02.405099 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:02.405013 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59bdd75cb8-f4lkw_4b9fa335-2643-477c-a8c2-155e4da1bdf4/manager/0.log" Jul 18 12:09:02.456857 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:02.456823 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tv2kp_98fa54ac-f9ae-40a0-aae7-617d6178e5fa/openshift-lws-operator/0.log" Jul 18 12:09:03.651690 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:03.651645 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7t8qd_bd5801a2-c674-4f9f-9122-74ec9f6eb6b0/discovery/0.log" Jul 18 12:09:05.158925 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:05.158804 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7t8qd_bd5801a2-c674-4f9f-9122-74ec9f6eb6b0/discovery/0.log" Jul 18 12:09:09.242068 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.241987 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4k74j_4af2d7f8-d600-497e-b76f-6853bf602faa/kube-multus/0.log" Jul 18 12:09:09.267042 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.267014 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bbjc_38ac6113-5bbc-4894-a994-2436d51392e3/kube-multus-additional-cni-plugins/0.log" Jul 18 12:09:09.286955 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.286925 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bbjc_38ac6113-5bbc-4894-a994-2436d51392e3/egress-router-binary-copy/0.log" Jul 18 12:09:09.310663 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.310635 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bbjc_38ac6113-5bbc-4894-a994-2436d51392e3/cni-plugins/0.log" Jul 18 12:09:09.332766 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.332724 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bbjc_38ac6113-5bbc-4894-a994-2436d51392e3/bond-cni-plugin/0.log" Jul 18 12:09:09.352517 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.352492 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bbjc_38ac6113-5bbc-4894-a994-2436d51392e3/routeoverride-cni/0.log" Jul 18 12:09:09.372755 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.372727 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bbjc_38ac6113-5bbc-4894-a994-2436d51392e3/whereabouts-cni-bincopy/0.log" Jul 18 12:09:09.391831 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.391800 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2bbjc_38ac6113-5bbc-4894-a994-2436d51392e3/whereabouts-cni/0.log" Jul 18 12:09:09.747226 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.747192 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bx2rv_ab36abb8-bb6e-458f-b909-277a53c37126/network-metrics-daemon/0.log" Jul 18 12:09:09.766159 ip-10-0-142-41 kubenswrapper[2585]: I0718 12:09:09.766128 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bx2rv_ab36abb8-bb6e-458f-b909-277a53c37126/kube-rbac-proxy/0.log"