Jul 23 18:06:29.108855 ip-10-0-128-128 systemd[1]: Starting Kubernetes Kubelet... Jul 23 18:06:31.820034 ip-10-0-128-128 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 18:06:31.820034 ip-10-0-128-128 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 18:06:31.820034 ip-10-0-128-128 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 18:06:31.820034 ip-10-0-128-128 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 18:06:31.820034 ip-10-0-128-128 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 18:06:31.820034 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.816179 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823140 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823158 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823164 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823169 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823174 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823178 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823184 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823188 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823192 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823196 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823200 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823204 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823208 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823212 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823216 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823219 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823223 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823227 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:06:31.825782 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823231 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823235 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823239 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823245 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823249 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823254 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823258 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823262 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823266 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823270 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823274 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823277 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823281 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823286 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823290 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823295 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823299 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823302 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823306 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823310 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:06:31.827135 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823314 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823320 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823325 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823330 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823334 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823337 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823342 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823346 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823350 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823354 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823358 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823377 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823382 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823385 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823390 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823395 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823400 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823403 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823408 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:06:31.831559 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823414 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823418 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823423 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823428 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823432 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823436 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823439 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823443 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823447 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823450 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823454 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823458 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823462 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823466 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823470 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823475 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823479 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823486 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823491 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823496 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:06:31.832293 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823501 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823505 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823509 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823513 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823517 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823521 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823525 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823529 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823533 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823536 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823540 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823544 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823547 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823551 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823555 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823559 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823564 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823567 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823571 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823575 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:06:31.833039 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823579 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.823583 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825343 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825355 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825361 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825383 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825388 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825393 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825398 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825402 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825406 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825410 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825414 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825418 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825422 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825426 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825430 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825434 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825441 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:06:31.835655 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825446 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825450 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825455 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825460 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825465 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825470 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825474 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825477 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825482 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825486 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825489 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825493 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825497 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825501 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825504 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825508 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825512 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825515 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825519 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:06:31.836354 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825522 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825526 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825531 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825538 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825542 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825545 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825549 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825553 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825557 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825561 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825565 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825568 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825572 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825576 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825580 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825583 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825587 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825590 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825594 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825598 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:06:31.947968 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825602 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:06:31.892166 ip-10-0-128-128 systemd[1]: Started Kubernetes Kubelet. Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825608 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825611 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825615 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825619 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825623 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825626 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825630 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825634 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825638 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825643 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825647 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825651 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825655 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825658 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825665 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825669 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825673 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825676 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825680 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:06:31.949060 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825684 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825688 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825691 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825695 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825699 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825702 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825708 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825712 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825716 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825719 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825723 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825727 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825731 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825740 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825744 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825747 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825751 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825755 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825758 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825762 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:06:31.951269 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825766 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825771 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.825774 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826406 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826416 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826423 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826428 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826434 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826438 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826443 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826447 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826451 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826455 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826459 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826463 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826466 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826470 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826473 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826477 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:06:31.952157 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826482 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826485 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826489 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826494 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826498 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826502 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826508 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826512 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826515 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826519 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826523 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826527 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826531 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826534 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826538 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826542 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826546 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826550 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826554 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826558 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:06:32.071624 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826563 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826569 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826574 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826578 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826582 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826586 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826590 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826594 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826597 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826602 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826605 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826609 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826613 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826617 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826620 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826624 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826628 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826631 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826636 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:06:32.072684 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826640 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826644 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826648 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826652 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826656 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826660 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826663 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826667 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826671 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826675 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826679 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826683 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826686 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826691 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826694 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826700 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826704 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826708 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826712 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826716 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:06:32.073440 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826720 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826724 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826727 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826731 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826735 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826739 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826744 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826748 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826751 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826756 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826759 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826765 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826768 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826772 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826776 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826780 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826784 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826788 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826792 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826795 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:06:32.074901 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826799 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826803 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826808 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.826814 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826925 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826936 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826949 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826957 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826970 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826976 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826982 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826989 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826994 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.826998 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827003 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827008 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827013 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827018 2555 flags.go:64] FLAG: --cgroup-root="" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827022 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827027 2555 flags.go:64] FLAG: --client-ca-file="" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827031 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827036 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827046 2555 flags.go:64] FLAG: --cluster-domain="" Jul 23 18:06:32.075658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827052 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827057 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827063 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827068 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827074 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827079 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827083 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827089 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827093 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827098 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827102 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827107 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827111 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827117 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827126 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827131 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827135 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827140 2555 flags.go:64] FLAG: --enable-server="true" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827152 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827162 2555 flags.go:64] FLAG: --event-burst="100" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827167 2555 flags.go:64] FLAG: --event-qps="50" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827171 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827175 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827180 2555 flags.go:64] FLAG: --eviction-hard="" Jul 23 18:06:32.076791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827186 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827191 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827195 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827200 2555 flags.go:64] FLAG: --eviction-soft="" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827204 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827208 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827213 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827217 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827224 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827228 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827232 2555 flags.go:64] FLAG: --feature-gates="" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827238 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827243 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827247 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827251 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827256 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827260 2555 flags.go:64] FLAG: --help="false" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827265 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827270 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827274 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827279 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827284 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827291 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827295 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 18:06:32.077509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827300 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827304 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827308 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827320 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827325 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827329 2555 flags.go:64] FLAG: --kube-reserved="" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827334 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827338 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827343 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827348 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827352 2555 flags.go:64] FLAG: --lock-file="" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827356 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827360 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827379 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827387 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827392 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827399 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827404 2555 flags.go:64] FLAG: --logging-format="text" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827408 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827413 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827418 2555 flags.go:64] FLAG: --manifest-url="" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827422 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827429 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827434 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827439 2555 flags.go:64] FLAG: --max-pods="110" Jul 23 18:06:32.078638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827444 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827448 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827453 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827457 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827462 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827467 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827471 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827488 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827493 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827497 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827502 2555 flags.go:64] FLAG: --pod-cidr="" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827507 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827516 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827520 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827525 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827529 2555 flags.go:64] FLAG: --port="10250" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827533 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827538 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02c683359735da844" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827543 2555 flags.go:64] FLAG: --qos-reserved="" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827547 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827552 2555 flags.go:64] FLAG: --register-node="true" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827557 2555 flags.go:64] FLAG: --register-with-taints="" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827563 2555 flags.go:64] FLAG: --registry-burst="10" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827567 2555 flags.go:64] FLAG: --registry-qps="5" Jul 23 18:06:32.080911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827573 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827578 2555 flags.go:64] FLAG: --reserved-memory="" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827584 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827589 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827593 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827598 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827602 2555 flags.go:64] FLAG: --runonce="false" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827607 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827611 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827616 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827620 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827625 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827629 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827636 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827641 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827646 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827650 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827654 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827658 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827663 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827668 2555 flags.go:64] FLAG: --system-cgroups="" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827673 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827682 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827686 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827690 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 18:06:32.081602 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827697 2555 flags.go:64] FLAG: --tls-min-version="" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827702 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827706 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827711 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827715 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827720 2555 flags.go:64] FLAG: --v="2" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827726 2555 flags.go:64] FLAG: --version="false" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827734 2555 flags.go:64] FLAG: --vmodule="" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827739 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.827745 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827950 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827957 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827963 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827967 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827971 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827975 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827979 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827983 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827987 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827991 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.827997 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828001 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:06:32.082212 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828005 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828009 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828013 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828017 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828021 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828025 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828030 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828034 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828039 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828043 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828047 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828051 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828055 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828058 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828062 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828066 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828070 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828075 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828079 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828083 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:06:32.082762 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828087 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828091 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828094 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828098 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828102 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828106 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828110 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828114 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828118 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828121 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828127 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828131 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828135 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828139 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828143 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828147 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828150 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828154 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828158 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828163 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:06:32.083236 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828167 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828171 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828177 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828182 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828186 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828189 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828195 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828200 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828204 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828210 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828214 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828218 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828222 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828226 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828230 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828233 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828239 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828243 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:06:32.083813 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828248 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828252 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828256 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828260 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828265 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828269 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828273 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828276 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828281 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828284 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828288 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828292 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828296 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828299 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828305 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828309 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828313 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828316 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828320 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828324 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:06:32.084250 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828328 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828332 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828336 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828341 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828345 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828349 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828353 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828357 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.828361 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.828917 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.836409 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.836432 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.837069 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.837077 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 18:06:32.084763 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837146 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837151 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837154 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837157 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837160 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837163 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837167 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837172 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837175 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837178 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837181 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837184 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837188 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837191 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837194 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837197 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837200 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837203 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837205 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:06:32.085116 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837208 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837211 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837214 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837216 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837219 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837221 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837224 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837226 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837229 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837232 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837241 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837243 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837246 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837249 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837251 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837254 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837256 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837259 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837261 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837264 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:06:32.085576 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837266 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837268 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837271 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837274 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837276 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837278 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837281 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837283 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837286 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837288 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837291 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837293 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837296 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837298 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837301 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837304 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837306 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837309 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837312 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:06:32.086051 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837314 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837317 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837320 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837323 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837331 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837333 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837336 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837339 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837341 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837344 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837346 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837349 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837352 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837354 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837356 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837359 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837374 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837381 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837384 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:06:32.086514 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837387 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837390 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837393 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837395 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837398 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837401 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837403 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837406 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837409 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837411 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837414 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837416 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837419 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837423 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837426 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837428 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837431 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837433 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837443 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837445 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837448 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:06:32.087109 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837451 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.837457 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837605 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837611 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837614 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837617 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837620 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837623 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837625 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837628 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837631 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837633 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837636 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837638 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837641 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 18:06:32.087992 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837643 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837646 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837648 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837651 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837655 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837659 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837663 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837666 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837669 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837671 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837674 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837676 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837679 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837682 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837684 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837692 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837695 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837698 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837700 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 18:06:32.088485 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837703 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837705 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837708 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837711 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837713 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837716 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837718 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837722 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837726 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837729 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837732 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837734 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837737 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837740 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837742 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837744 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837747 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837749 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837752 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837755 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 18:06:32.088960 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837757 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837760 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837763 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837765 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837767 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837770 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837772 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837775 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837777 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837785 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837788 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837791 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837794 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837798 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837801 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837804 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837806 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837809 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837812 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837817 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 18:06:32.089459 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837820 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837823 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837826 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837828 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837831 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837839 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837842 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837844 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837847 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837849 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837852 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837854 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837857 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837859 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837862 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837864 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837866 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837869 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837872 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837874 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 18:06:32.089939 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837877 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837879 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837888 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837891 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837894 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837896 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:31.837898 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.837903 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.838063 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.842276 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.843618 2555 server.go:1019] "Starting client certificate rotation" Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.843721 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.843760 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 18:06:32.090421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.845867 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 18:06:32.090798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.847118 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 18:06:32.090798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.858066 2555 log.go:25] "Validated CRI v1 runtime API" Jul 23 18:06:32.090798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.860436 2555 log.go:25] "Validated CRI v1 image API" Jul 23 18:06:32.090798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.861295 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 18:06:32.090798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.863477 2555 fs.go:135] Filesystem UUIDs: map[1081b409-c111-4030-a2de-748919b9accc:/dev/nvme0n1p3 1f9967aa-2a35-4f7f-9ebf-ac4f9ac9f212:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 23 18:06:32.090798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.863495 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.868360 2555 manager.go:216] Machine: {Timestamp:2026-07-23 18:06:31.867852406 +0000 UTC m=+1.564973471 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3085286 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22e8e8890fc7a47f297a555618f61e SystemUUID:ec22e8e8-890f-c7a4-7f29-7a555618f61e BootID:8ef99dbe-4198-4c1a-8bda-4dba360723a3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:65:5f:c4:3f:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:65:5f:c4:3f:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:dc:4a:79:88:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.868732 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.868870 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.871561 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.871591 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-128.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.872276 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.872285 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.872293 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.872900 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.874111 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.877103 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.877246 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.877836 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.877852 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.877867 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.877877 2555 kubelet.go:398] "Adding apiserver pod source" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.877888 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 18:06:32.090951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.878276 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.878293 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.884227 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.886174 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.886208 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887214 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887229 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887235 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887240 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887248 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887254 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887261 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887267 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887275 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887280 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887289 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887294 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887894 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.887901 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.890211 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.891191 2555 server.go:1268] "Started kubelet" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.891316 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.891403 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.891755 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.894449 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-128.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.894469 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.894509 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.895398 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.895751 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.897744 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-128.ec2.internal.18c4fd6bd322cb9f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-128.ec2.internal,UID:ip-10-0-128-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-128.ec2.internal,},FirstTimestamp:2026-07-23 18:06:31.891159967 +0000 UTC m=+1.588281105,LastTimestamp:2026-07-23 18:06:31.891159967 +0000 UTC m=+1.588281105,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-128.ec2.internal,}" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.900993 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901105 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901120 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.901198 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901748 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901758 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901762 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901872 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901881 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901945 2555 factory.go:55] Registering systemd factory Jul 23 18:06:32.091443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.901961 2555 factory.go:223] Registration of the systemd container factory successfully Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.902175 2555 factory.go:153] Registering CRI-O factory Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.902189 2555 factory.go:223] Registration of the crio container factory successfully Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.902243 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.902269 2555 factory.go:103] Registering Raw factory Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.902284 2555 manager.go:1196] Started watching for new ooms in manager Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.902750 2555 manager.go:349] Starting recovery of all containers Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.909610 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.915669 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.919393 2555 manager.go:354] Recovery completed Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.924204 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.926480 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.926514 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.926528 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.927783 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.927798 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.927812 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.927822 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.928573 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.928613 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.928626 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.928733 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-128.ec2.internal.18c4fd6bd53e0479 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-128.ec2.internal,UID:ip-10-0-128-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-128.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-128.ec2.internal,},FirstTimestamp:2026-07-23 18:06:31.926498425 +0000 UTC m=+1.623619491,LastTimestamp:2026-07-23 18:06:31.926498425 +0000 UTC m=+1.623619491,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-128.ec2.internal,}" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.929414 2555 policy_none.go:49] "None policy: Start" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.929428 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.929438 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.930418 2555 policy_none.go:47] "Start" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.934724 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l79cv" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.938892 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-128.ec2.internal.18c4fd6bd53e54a1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-128.ec2.internal,UID:ip-10-0-128-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-128.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-128.ec2.internal,},FirstTimestamp:2026-07-23 18:06:31.926518945 +0000 UTC m=+1.623640011,LastTimestamp:2026-07-23 18:06:31.926518945 +0000 UTC m=+1.623640011,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-128.ec2.internal,}" Jul 23 18:06:32.092436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.956202 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l79cv" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.964558 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-128.ec2.internal.18c4fd6bd53e8939 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-128.ec2.internal,UID:ip-10-0-128-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-128.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-128.ec2.internal,},FirstTimestamp:2026-07-23 18:06:31.926532409 +0000 UTC m=+1.623653474,LastTimestamp:2026-07-23 18:06:31.926532409 +0000 UTC m=+1.623653474,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-128.ec2.internal,}" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.968065 2555 manager.go:337] "Starting Device Plugin manager" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.968093 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.968103 2555 server.go:87] "Starting device plugin registration server" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.968391 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.968403 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.968467 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.968540 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:31.968546 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.969075 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:31.969112 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.068727 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.069786 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.069836 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.069848 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.069879 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.080922 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.093142 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.080948 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-128.ec2.internal\": node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.095948 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.095929 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.196657 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.196618 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.239527 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.239446 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 18:06:32.240897 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.240872 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 18:06:32.241012 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.240922 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 18:06:32.241012 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.240947 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 18:06:32.241012 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.240988 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 18:06:32.247633 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.247610 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:06:32.297404 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.297377 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.341909 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.341867 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal"] Jul 23 18:06:32.342014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.341988 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:32.343023 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.343006 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:32.343099 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.343041 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:32.343099 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.343052 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:32.345435 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.345423 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:32.345530 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.345513 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.345567 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.345550 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:32.346220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.346205 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:32.346276 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.346229 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:32.346276 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.346238 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:32.346276 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.346259 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:32.346426 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.346280 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:32.346426 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.346298 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:32.348571 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.348553 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.348685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.348579 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:32.350807 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.350789 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:32.350906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.350822 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:32.350906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.350836 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:32.372302 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.372272 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-128.ec2.internal\" not found" node="ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.377426 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.377405 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-128.ec2.internal\" not found" node="ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.398126 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.398081 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.405743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.405711 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0869f205358e1c1285fb845001c950b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal\" (UID: \"f0869f205358e1c1285fb845001c950b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.405893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.405749 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0869f205358e1c1285fb845001c950b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal\" (UID: \"f0869f205358e1c1285fb845001c950b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.405893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.405783 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b519a6809738d51a152b4478c04773cc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-128.ec2.internal\" (UID: \"b519a6809738d51a152b4478c04773cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.498259 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.498178 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.506395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.506354 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b519a6809738d51a152b4478c04773cc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-128.ec2.internal\" (UID: \"b519a6809738d51a152b4478c04773cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.506489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.506400 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0869f205358e1c1285fb845001c950b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal\" (UID: \"f0869f205358e1c1285fb845001c950b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.506489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.506418 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0869f205358e1c1285fb845001c950b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal\" (UID: \"f0869f205358e1c1285fb845001c950b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.506489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.506465 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0869f205358e1c1285fb845001c950b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal\" (UID: \"f0869f205358e1c1285fb845001c950b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.506489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.506476 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b519a6809738d51a152b4478c04773cc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-128.ec2.internal\" (UID: \"b519a6809738d51a152b4478c04773cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.506631 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.506479 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0869f205358e1c1285fb845001c950b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal\" (UID: \"f0869f205358e1c1285fb845001c950b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.598506 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.598473 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.676073 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.676051 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.681316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.680909 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:32.699065 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.698830 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.787019 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.786932 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:06:32.799793 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.799756 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.843235 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.843210 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 18:06:32.843671 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.843390 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 18:06:32.843671 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.843397 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 18:06:32.900104 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:32.900073 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:32.963987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.963940 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 18:01:31 +0000 UTC" deadline="2028-05-08 23:40:50.011205442 +0000 UTC" Jul 23 18:06:32.963987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:32.963977 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15725h34m17.047232574s" Jul 23 18:06:33.000309 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:33.000277 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:33.100886 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:33.100787 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-128.ec2.internal\" not found" Jul 23 18:06:33.120220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.120191 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 18:06:33.121957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.121935 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientMemory" Jul 23 18:06:33.122094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.121970 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 18:06:33.122094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.121981 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeHasSufficientPID" Jul 23 18:06:33.166252 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.166220 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:06:33.201565 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.201529 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" Jul 23 18:06:33.214823 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.214803 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 18:06:33.215612 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.215598 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" Jul 23 18:06:33.220469 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:33.220435 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0869f205358e1c1285fb845001c950b.slice/crio-3867f06fa8098f62acef57a688948417812fbd33daf1929d31668b1ea37c1c4b WatchSource:0}: Error finding container 3867f06fa8098f62acef57a688948417812fbd33daf1929d31668b1ea37c1c4b: Status 404 returned error can't find the container with id 3867f06fa8098f62acef57a688948417812fbd33daf1929d31668b1ea37c1c4b Jul 23 18:06:33.225851 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.225834 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 18:06:33.225923 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.225885 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:06:33.246002 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.245952 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" event={"ID":"f0869f205358e1c1285fb845001c950b","Type":"ContainerStarted","Data":"3867f06fa8098f62acef57a688948417812fbd33daf1929d31668b1ea37c1c4b"} Jul 23 18:06:33.254068 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:33.254041 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb519a6809738d51a152b4478c04773cc.slice/crio-ad68dc3bf6cd39b071f3a646333dc9f1530d792d1422bffc111b8caef23aff0f WatchSource:0}: Error finding container ad68dc3bf6cd39b071f3a646333dc9f1530d792d1422bffc111b8caef23aff0f: Status 404 returned error can't find the container with id ad68dc3bf6cd39b071f3a646333dc9f1530d792d1422bffc111b8caef23aff0f Jul 23 18:06:33.266291 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.266270 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:06:33.879783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.879749 2555 apiserver.go:52] "Watching apiserver" Jul 23 18:06:33.890664 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.889634 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 18:06:33.890664 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.890289 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-ldl42","openshift-network-diagnostics/network-check-target-mlbqg","openshift-ovn-kubernetes/ovnkube-node-qjdpg","kube-system/global-pull-secret-syncer-tlbgv","openshift-cluster-node-tuning-operator/tuned-srsjc","openshift-image-registry/node-ca-wss72","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal","openshift-multus/multus-5sfpq","openshift-network-operator/iptables-alerter-8jzs5","kube-system/konnectivity-agent-5n6tt","kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4","openshift-dns/node-resolver-2tj8f","openshift-multus/multus-additional-cni-plugins-n28cd"] Jul 23 18:06:33.892821 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.892794 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:33.895797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.895778 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:33.896407 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.896192 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zszvg\"" Jul 23 18:06:33.897397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.897216 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 18:06:33.897397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.897262 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 18:06:33.898491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.898194 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gvxct\"" Jul 23 18:06:33.898491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.898244 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 18:06:33.898643 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.898528 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.901187 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.901033 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 18:06:33.901187 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.901137 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:33.901384 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:33.901204 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:33.902605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.901695 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 18:06:33.902605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.901719 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 18:06:33.902605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.902155 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 18:06:33.902605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.902399 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jxgdx\"" Jul 23 18:06:33.902605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.902435 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 18:06:33.904693 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.903043 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.904693 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.903320 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 18:06:33.904693 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.903598 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 18:06:33.904693 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.903674 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 18:06:33.905716 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.905694 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:06:33.905945 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.905909 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-96ddn\"" Jul 23 18:06:33.906031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.905910 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:33.906031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.905987 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 18:06:33.908201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.907842 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:33.910559 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.910540 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:33.911921 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.911900 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 18:06:33.912122 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.912103 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 18:06:33.912303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.912286 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 18:06:33.913007 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.912612 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 18:06:33.913007 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.912799 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:06:33.913195 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913173 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m9z8f\"" Jul 23 18:06:33.913249 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913195 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:33.913314 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:33.913262 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:33.913397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913382 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vrmgl\"" Jul 23 18:06:33.913618 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913599 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913685 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmw49\" (UniqueName: \"kubernetes.io/projected/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-kube-api-access-cmw49\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913747 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysctl-d\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913774 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-slash\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-run-netns\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-systemd\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913874 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-var-lib-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913898 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-node-log\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.913957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913945 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913971 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3491db21-f1c1-4f95-8eb5-95938c6657e6-tmp\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.913999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a56fdb46-cda2-4a61-afbd-5b46df9a57c3-konnectivity-ca\") pod \"konnectivity-agent-5n6tt\" (UID: \"a56fdb46-cda2-4a61-afbd-5b46df9a57c3\") " pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914023 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-kubelet\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914062 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914117 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysconfig\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914162 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-systemd\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914212 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-lib-modules\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914232 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-ovn\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914265 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-kubelet-config\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914294 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-kubernetes\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914316 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysctl-conf\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914337 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-run\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914396 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-tuned\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914420 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-env-overrides\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914443 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovn-node-metrics-cert\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914464 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-hosts-file\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:33.914489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914486 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-tmp-dir\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914511 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-cni-bin\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914581 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-cni-netd\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914646 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovnkube-config\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pnww\" (UniqueName: \"kubernetes.io/projected/3491db21-f1c1-4f95-8eb5-95938c6657e6-kube-api-access-6pnww\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914704 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovnkube-script-lib\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914730 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fvg8\" (UniqueName: \"kubernetes.io/projected/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-kube-api-access-5fvg8\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-dbus\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914795 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-var-lib-kubelet\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914818 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-host\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914856 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a56fdb46-cda2-4a61-afbd-5b46df9a57c3-agent-certs\") pod \"konnectivity-agent-5n6tt\" (UID: \"a56fdb46-cda2-4a61-afbd-5b46df9a57c3\") " pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914879 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914913 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914937 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-modprobe-d\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914960 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-sys\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.914997 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-systemd-units\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.915381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.915068 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-etc-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.916065 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.915098 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-log-socket\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:33.918160 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.916719 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:33.918160 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:33.916806 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:33.918160 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.917281 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 18:06:33.918160 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.917697 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 18:06:33.918160 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.918131 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 18:06:33.918470 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.918129 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5985h\"" Jul 23 18:06:33.918470 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.918300 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 18:06:33.919848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.918901 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 18:06:33.921654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.921634 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5sfpq" Jul 23 18:06:33.924763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.924742 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:33.929913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.929678 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qzh48\"" Jul 23 18:06:33.930112 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.930092 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 18:06:33.930608 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.930492 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 18:06:33.931405 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.931033 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 18:06:33.931405 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.931213 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 18:06:33.932408 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.932380 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v96p8\"" Jul 23 18:06:33.935235 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.935217 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 18:06:33.945079 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.945053 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-htnlm" Jul 23 18:06:33.956863 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:33.956439 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-htnlm" Jul 23 18:06:34.003624 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.003591 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 18:06:34.015631 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015597 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv7wt\" (UniqueName: \"kubernetes.io/projected/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-kube-api-access-vv7wt\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.015631 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015645 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-var-lib-kubelet\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.015872 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015671 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-os-release\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.015872 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015738 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-var-lib-kubelet\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.015872 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015791 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a56fdb46-cda2-4a61-afbd-5b46df9a57c3-agent-certs\") pod \"konnectivity-agent-5n6tt\" (UID: \"a56fdb46-cda2-4a61-afbd-5b46df9a57c3\") " pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:34.015872 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015830 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:34.015872 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015853 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-modprobe-d\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.016098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.015973 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-modprobe-d\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.016098 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.016063 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:34.016188 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.016138 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret podName:1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:34.516113432 +0000 UTC m=+4.213234507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret") pod "global-pull-secret-syncer-tlbgv" (UID: "1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:34.016279 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016229 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 18:06:34.016279 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016258 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2vfb\" (UniqueName: \"kubernetes.io/projected/3e36dcf4-2a4f-43af-8358-851a13af2f03-kube-api-access-j2vfb\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.016401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016287 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmw49\" (UniqueName: \"kubernetes.io/projected/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-kube-api-access-cmw49\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.016401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016320 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-cnibin\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.016401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-daemon-config\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.016401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016351 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42655\" (UniqueName: \"kubernetes.io/projected/28429fa2-bd08-431a-9c1f-7d191d397d01-kube-api-access-42655\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.016401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016388 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4gkb\" (UniqueName: \"kubernetes.io/projected/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-kube-api-access-z4gkb\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016411 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-run-netns\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-systemd\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016443 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-var-lib-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-node-log\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016485 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016501 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3491db21-f1c1-4f95-8eb5-95938c6657e6-tmp\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-cni-bin\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016540 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysconfig\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016559 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-etc-kubernetes\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016575 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-lib-modules\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.016652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016637 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-var-lib-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016677 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016693 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-lib-modules\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016707 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-kubelet\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016731 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-node-log\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016733 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-device-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016758 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016761 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-ovn\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016785 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-kubelet-config\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016810 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysctl-conf\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016847 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-tuned\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016872 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016897 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-cni-multus\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016920 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-hostroot\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016944 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e71b11c-141b-467a-9e9d-deb696739f55-serviceca\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016969 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-env-overrides\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016992 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysconfig\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.017162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.016993 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-cni-bin\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017029 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-cni-bin\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017040 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovnkube-config\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017066 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pnww\" (UniqueName: \"kubernetes.io/projected/3491db21-f1c1-4f95-8eb5-95938c6657e6-kube-api-access-6pnww\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017165 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cnibin\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017291 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-systemd\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017292 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017329 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-run-netns\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017388 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-ovn\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017416 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-kubelet-config\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017499 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysctl-conf\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017532 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-system-cni-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017583 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-netns\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017620 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovnkube-script-lib\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017676 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fvg8\" (UniqueName: \"kubernetes.io/projected/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-kube-api-access-5fvg8\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-dbus\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017850 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-env-overrides\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.017954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017908 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-host\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017951 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-cni-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017982 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-dbus\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.017982 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018032 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-host\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018052 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-sys\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018075 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-run-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovnkube-config\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018120 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e36dcf4-2a4f-43af-8358-851a13af2f03-host-slash\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018151 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-system-cni-dir\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018179 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-sys\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018200 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018231 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-k8s-cni-cncf-io\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t76gt\" (UniqueName: \"kubernetes.io/projected/1e71b11c-141b-467a-9e9d-deb696739f55-kube-api-access-t76gt\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018311 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-registration-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018344 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-systemd-units\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.018783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-etc-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-log-socket\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018444 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysctl-d\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018454 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-systemd-units\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018483 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-os-release\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018507 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018534 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjmtr\" (UniqueName: \"kubernetes.io/projected/80f84c0a-f11e-43de-bd24-d9d5abba846d-kube-api-access-fjmtr\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018535 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-sysctl-d\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018583 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-log-socket\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018622 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-etc-openvswitch\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018652 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-slash\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018687 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-multus-certs\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018711 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovnkube-script-lib\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018740 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-slash\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018765 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018805 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a56fdb46-cda2-4a61-afbd-5b46df9a57c3-konnectivity-ca\") pod \"konnectivity-agent-5n6tt\" (UID: \"a56fdb46-cda2-4a61-afbd-5b46df9a57c3\") " pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018838 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-kubelet\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.019600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.018881 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-kubelet\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019326 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a56fdb46-cda2-4a61-afbd-5b46df9a57c3-konnectivity-ca\") pod \"konnectivity-agent-5n6tt\" (UID: \"a56fdb46-cda2-4a61-afbd-5b46df9a57c3\") " pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019389 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019429 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-socket-dir-parent\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019457 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019489 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-socket-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019518 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-systemd\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019552 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3e36dcf4-2a4f-43af-8358-851a13af2f03-iptables-alerter-script\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019576 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-conf-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e71b11c-141b-467a-9e9d-deb696739f55-host\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019625 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-sys-fs\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019652 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-kubernetes\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019707 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-run\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019731 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovn-node-metrics-cert\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019757 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-hosts-file\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019781 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-tmp-dir\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019805 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-cni-netd\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.020410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.019832 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28429fa2-bd08-431a-9c1f-7d191d397d01-cni-binary-copy\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020011 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-kubernetes\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020032 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-run\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020063 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-systemd\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020097 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020112 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-hosts-file\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020115 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3491db21-f1c1-4f95-8eb5-95938c6657e6-tmp\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020179 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-host-cni-netd\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020359 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-tmp-dir\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:34.021303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.020858 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a56fdb46-cda2-4a61-afbd-5b46df9a57c3-agent-certs\") pod \"konnectivity-agent-5n6tt\" (UID: \"a56fdb46-cda2-4a61-afbd-5b46df9a57c3\") " pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:34.023188 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.023164 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-ovn-node-metrics-cert\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.023764 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.023740 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3491db21-f1c1-4f95-8eb5-95938c6657e6-etc-tuned\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.033000 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.032918 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmw49\" (UniqueName: \"kubernetes.io/projected/ab021d97-ecab-41fd-8d6b-5e01f0963e4f-kube-api-access-cmw49\") pod \"ovnkube-node-qjdpg\" (UID: \"ab021d97-ecab-41fd-8d6b-5e01f0963e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.034992 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.034969 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pnww\" (UniqueName: \"kubernetes.io/projected/3491db21-f1c1-4f95-8eb5-95938c6657e6-kube-api-access-6pnww\") pod \"tuned-srsjc\" (UID: \"3491db21-f1c1-4f95-8eb5-95938c6657e6\") " pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.035190 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.035158 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fvg8\" (UniqueName: \"kubernetes.io/projected/7e3d2607-3f4d-4eac-a069-ca8ae834bc85-kube-api-access-5fvg8\") pod \"node-resolver-2tj8f\" (UID: \"7e3d2607-3f4d-4eac-a069-ca8ae834bc85\") " pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:34.121131 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121086 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-hostroot\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121131 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121130 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e71b11c-141b-467a-9e9d-deb696739f55-serviceca\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.121392 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121201 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-hostroot\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121392 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cnibin\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.121392 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121325 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.121553 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-system-cni-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121553 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121467 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cnibin\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.121553 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121476 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-netns\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121553 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121501 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-cni-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121553 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121527 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-system-cni-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121567 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-netns\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121587 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e36dcf4-2a4f-43af-8358-851a13af2f03-host-slash\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-system-cni-dir\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121621 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-cni-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121636 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121656 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e36dcf4-2a4f-43af-8358-851a13af2f03-host-slash\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121661 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-k8s-cni-cncf-io\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121663 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e71b11c-141b-467a-9e9d-deb696739f55-serviceca\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121685 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t76gt\" (UniqueName: \"kubernetes.io/projected/1e71b11c-141b-467a-9e9d-deb696739f55-kube-api-access-t76gt\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121722 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-registration-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121755 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-k8s-cni-cncf-io\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121816 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-registration-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121817 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.121845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-os-release\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121859 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121888 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-os-release\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121888 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjmtr\" (UniqueName: \"kubernetes.io/projected/80f84c0a-f11e-43de-bd24-d9d5abba846d-kube-api-access-fjmtr\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121938 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-multus-certs\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121957 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-system-cni-dir\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121973 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.121986 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122001 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-socket-dir-parent\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122007 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122049 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122077 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-socket-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122101 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3e36dcf4-2a4f-43af-8358-851a13af2f03-iptables-alerter-script\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122116 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122108 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-run-multus-certs\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122162 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-socket-dir-parent\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122214 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-conf-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.122546 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.122228 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122237 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-socket-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122273 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e71b11c-141b-467a-9e9d-deb696739f55-host\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e71b11c-141b-467a-9e9d-deb696739f55-host\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122278 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-conf-dir\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.122308 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs podName:f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:34.622265001 +0000 UTC m=+4.319386068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs") pod "network-metrics-daemon-ldl42" (UID: "f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122350 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-sys-fs\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122401 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28429fa2-bd08-431a-9c1f-7d191d397d01-cni-binary-copy\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122429 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv7wt\" (UniqueName: \"kubernetes.io/projected/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-kube-api-access-vv7wt\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122457 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-os-release\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122491 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-sys-fs\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122499 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2vfb\" (UniqueName: \"kubernetes.io/projected/3e36dcf4-2a4f-43af-8358-851a13af2f03-kube-api-access-j2vfb\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122541 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-cnibin\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122565 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-daemon-config\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122590 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42655\" (UniqueName: \"kubernetes.io/projected/28429fa2-bd08-431a-9c1f-7d191d397d01-kube-api-access-42655\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122617 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4gkb\" (UniqueName: \"kubernetes.io/projected/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-kube-api-access-z4gkb\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122645 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-cni-bin\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.123345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122668 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-os-release\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-etc-kubernetes\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122717 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3e36dcf4-2a4f-43af-8358-851a13af2f03-iptables-alerter-script\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122733 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-cnibin\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122742 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-kubelet\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122776 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-cni-bin\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122794 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-device-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122855 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122885 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-cni-multus\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-cni-multus\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.122997 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-host-var-lib-kubelet\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.123067 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80f84c0a-f11e-43de-bd24-d9d5abba846d-device-dir\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.123101 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28429fa2-bd08-431a-9c1f-7d191d397d01-etc-kubernetes\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.123108 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28429fa2-bd08-431a-9c1f-7d191d397d01-cni-binary-copy\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.123231 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/28429fa2-bd08-431a-9c1f-7d191d397d01-multus-daemon-config\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.123553 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.124211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.123587 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.137325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.136158 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t76gt\" (UniqueName: \"kubernetes.io/projected/1e71b11c-141b-467a-9e9d-deb696739f55-kube-api-access-t76gt\") pod \"node-ca-wss72\" (UID: \"1e71b11c-141b-467a-9e9d-deb696739f55\") " pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.137325 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.136169 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:06:34.137325 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.136241 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:06:34.137325 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.136256 2555 projected.go:196] Error preparing data for projected volume kube-api-access-6svd9 for pod openshift-network-diagnostics/network-check-target-mlbqg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:34.137325 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.136331 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9 podName:53440927-f60b-4282-a510-d08f54e11f45 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:34.636311483 +0000 UTC m=+4.333432549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6svd9" (UniqueName: "kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9") pod "network-check-target-mlbqg" (UID: "53440927-f60b-4282-a510-d08f54e11f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:34.139077 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.139044 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv7wt\" (UniqueName: \"kubernetes.io/projected/ad9d3a05-d68e-4fb4-9dff-6333cb8420ec-kube-api-access-vv7wt\") pod \"multus-additional-cni-plugins-n28cd\" (UID: \"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec\") " pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.140102 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.140076 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4gkb\" (UniqueName: \"kubernetes.io/projected/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-kube-api-access-z4gkb\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:34.140214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.140160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42655\" (UniqueName: \"kubernetes.io/projected/28429fa2-bd08-431a-9c1f-7d191d397d01-kube-api-access-42655\") pod \"multus-5sfpq\" (UID: \"28429fa2-bd08-431a-9c1f-7d191d397d01\") " pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.143032 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.142947 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjmtr\" (UniqueName: \"kubernetes.io/projected/80f84c0a-f11e-43de-bd24-d9d5abba846d-kube-api-access-fjmtr\") pod \"aws-ebs-csi-driver-node-tmdh4\" (UID: \"80f84c0a-f11e-43de-bd24-d9d5abba846d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.154526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.154419 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2vfb\" (UniqueName: \"kubernetes.io/projected/3e36dcf4-2a4f-43af-8358-851a13af2f03-kube-api-access-j2vfb\") pod \"iptables-alerter-8jzs5\" (UID: \"3e36dcf4-2a4f-43af-8358-851a13af2f03\") " pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.210494 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.210456 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:34.223651 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.223620 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2tj8f" Jul 23 18:06:34.231408 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.231381 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:06:34.238037 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.238012 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-srsjc" Jul 23 18:06:34.246254 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.246191 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wss72" Jul 23 18:06:34.248169 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.248101 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" event={"ID":"b519a6809738d51a152b4478c04773cc","Type":"ContainerStarted","Data":"ad68dc3bf6cd39b071f3a646333dc9f1530d792d1422bffc111b8caef23aff0f"} Jul 23 18:06:34.254876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.254844 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8jzs5" Jul 23 18:06:34.262737 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.262706 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n28cd" Jul 23 18:06:34.270671 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.270646 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5sfpq" Jul 23 18:06:34.277537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.277511 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" Jul 23 18:06:34.337271 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.337241 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:06:34.525926 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.525824 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:34.526070 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.526014 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:34.526130 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.526099 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret podName:1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:35.526075549 +0000 UTC m=+5.223196603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret") pod "global-pull-secret-syncer-tlbgv" (UID: "1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:34.626660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.626617 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:34.626813 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.626774 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:34.626876 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.626857 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs podName:f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:35.626835476 +0000 UTC m=+5.323956553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs") pod "network-metrics-daemon-ldl42" (UID: "f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:34.727606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.727563 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:34.727807 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.727720 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:06:34.727807 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.727739 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:06:34.727807 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.727752 2555 projected.go:196] Error preparing data for projected volume kube-api-access-6svd9 for pod openshift-network-diagnostics/network-check-target-mlbqg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:34.727961 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:34.727819 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9 podName:53440927-f60b-4282-a510-d08f54e11f45 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:35.727800088 +0000 UTC m=+5.424921140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6svd9" (UniqueName: "kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9") pod "network-check-target-mlbqg" (UID: "53440927-f60b-4282-a510-d08f54e11f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:34.960797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.960695 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:01:33 +0000 UTC" deadline="2028-03-07 19:27:52.751388738 +0000 UTC" Jul 23 18:06:34.960797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:34.960738 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14233h21m17.790655952s" Jul 23 18:06:35.133465 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.133433 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e36dcf4_2a4f_43af_8358_851a13af2f03.slice/crio-6e46bf165f42707ee9c0e2248b7044e2b136de10f6d6edc43bf9549886aaea19 WatchSource:0}: Error finding container 6e46bf165f42707ee9c0e2248b7044e2b136de10f6d6edc43bf9549886aaea19: Status 404 returned error can't find the container with id 6e46bf165f42707ee9c0e2248b7044e2b136de10f6d6edc43bf9549886aaea19 Jul 23 18:06:35.134306 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.134266 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3491db21_f1c1_4f95_8eb5_95938c6657e6.slice/crio-00d497208ab0d3f65175ba1b9d5608ffb2ac4964110f374a9bf80176afe076bd WatchSource:0}: Error finding container 00d497208ab0d3f65175ba1b9d5608ffb2ac4964110f374a9bf80176afe076bd: Status 404 returned error can't find the container with id 00d497208ab0d3f65175ba1b9d5608ffb2ac4964110f374a9bf80176afe076bd Jul 23 18:06:35.135889 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.135822 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9d3a05_d68e_4fb4_9dff_6333cb8420ec.slice/crio-520074566b9d99bfe8b87214b492092aae3555a0435aa8a0eef254ffd6945d79 WatchSource:0}: Error finding container 520074566b9d99bfe8b87214b492092aae3555a0435aa8a0eef254ffd6945d79: Status 404 returned error can't find the container with id 520074566b9d99bfe8b87214b492092aae3555a0435aa8a0eef254ffd6945d79 Jul 23 18:06:35.137352 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.137245 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28429fa2_bd08_431a_9c1f_7d191d397d01.slice/crio-adb917420561bdb028574d997e0dd656c8f745c7e40d86c862327d14fc978468 WatchSource:0}: Error finding container adb917420561bdb028574d997e0dd656c8f745c7e40d86c862327d14fc978468: Status 404 returned error can't find the container with id adb917420561bdb028574d997e0dd656c8f745c7e40d86c862327d14fc978468 Jul 23 18:06:35.141068 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.140765 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f84c0a_f11e_43de_bd24_d9d5abba846d.slice/crio-6ab66aa97448e1888ed32725bed9fa3f7c8ce19807e6b837c220456186676d9c WatchSource:0}: Error finding container 6ab66aa97448e1888ed32725bed9fa3f7c8ce19807e6b837c220456186676d9c: Status 404 returned error can't find the container with id 6ab66aa97448e1888ed32725bed9fa3f7c8ce19807e6b837c220456186676d9c Jul 23 18:06:35.141409 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.141386 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56fdb46_cda2_4a61_afbd_5b46df9a57c3.slice/crio-2a101513271f56c00a7bef0d741e0e5a3d85cd7fa91f881423bb4bb4e68fe267 WatchSource:0}: Error finding container 2a101513271f56c00a7bef0d741e0e5a3d85cd7fa91f881423bb4bb4e68fe267: Status 404 returned error can't find the container with id 2a101513271f56c00a7bef0d741e0e5a3d85cd7fa91f881423bb4bb4e68fe267 Jul 23 18:06:35.141983 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.141885 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab021d97_ecab_41fd_8d6b_5e01f0963e4f.slice/crio-5ae9bbde625bf3c28413f628d1617b86956794637d6869c2f36e5552a005572b WatchSource:0}: Error finding container 5ae9bbde625bf3c28413f628d1617b86956794637d6869c2f36e5552a005572b: Status 404 returned error can't find the container with id 5ae9bbde625bf3c28413f628d1617b86956794637d6869c2f36e5552a005572b Jul 23 18:06:35.143208 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.143166 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e71b11c_141b_467a_9e9d_deb696739f55.slice/crio-81ec8b23f5873561cb416233dfa019c0459f0affb1ab8299b9a778a5a2ff2fa0 WatchSource:0}: Error finding container 81ec8b23f5873561cb416233dfa019c0459f0affb1ab8299b9a778a5a2ff2fa0: Status 404 returned error can't find the container with id 81ec8b23f5873561cb416233dfa019c0459f0affb1ab8299b9a778a5a2ff2fa0 Jul 23 18:06:35.145405 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:06:35.145343 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3d2607_3f4d_4eac_a069_ca8ae834bc85.slice/crio-563f239270479f3f1804fd26586d5ea2711bde3a80ad2e86dc7f50867a24ca88 WatchSource:0}: Error finding container 563f239270479f3f1804fd26586d5ea2711bde3a80ad2e86dc7f50867a24ca88: Status 404 returned error can't find the container with id 563f239270479f3f1804fd26586d5ea2711bde3a80ad2e86dc7f50867a24ca88 Jul 23 18:06:35.241967 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.241760 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:35.242134 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.242073 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:35.242134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.241771 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:35.242134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.241762 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:35.242251 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.242169 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:35.242298 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.242274 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:35.250931 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.250892 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8jzs5" event={"ID":"3e36dcf4-2a4f-43af-8358-851a13af2f03","Type":"ContainerStarted","Data":"6e46bf165f42707ee9c0e2248b7044e2b136de10f6d6edc43bf9549886aaea19"} Jul 23 18:06:35.253031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.253001 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" event={"ID":"b519a6809738d51a152b4478c04773cc","Type":"ContainerStarted","Data":"150b2cc58f83b58d5e25e7fb1b72cb69dbfa195c2ed81a4a1f8ceb8c044e45b4"} Jul 23 18:06:35.254280 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.254256 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2tj8f" event={"ID":"7e3d2607-3f4d-4eac-a069-ca8ae834bc85","Type":"ContainerStarted","Data":"563f239270479f3f1804fd26586d5ea2711bde3a80ad2e86dc7f50867a24ca88"} Jul 23 18:06:35.255467 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.255446 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wss72" event={"ID":"1e71b11c-141b-467a-9e9d-deb696739f55","Type":"ContainerStarted","Data":"81ec8b23f5873561cb416233dfa019c0459f0affb1ab8299b9a778a5a2ff2fa0"} Jul 23 18:06:35.256484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.256463 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" event={"ID":"80f84c0a-f11e-43de-bd24-d9d5abba846d","Type":"ContainerStarted","Data":"6ab66aa97448e1888ed32725bed9fa3f7c8ce19807e6b837c220456186676d9c"} Jul 23 18:06:35.257655 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.257635 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5sfpq" event={"ID":"28429fa2-bd08-431a-9c1f-7d191d397d01","Type":"ContainerStarted","Data":"adb917420561bdb028574d997e0dd656c8f745c7e40d86c862327d14fc978468"} Jul 23 18:06:35.260039 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.260006 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerStarted","Data":"520074566b9d99bfe8b87214b492092aae3555a0435aa8a0eef254ffd6945d79"} Jul 23 18:06:35.261925 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.261902 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-srsjc" event={"ID":"3491db21-f1c1-4f95-8eb5-95938c6657e6","Type":"ContainerStarted","Data":"00d497208ab0d3f65175ba1b9d5608ffb2ac4964110f374a9bf80176afe076bd"} Jul 23 18:06:35.263383 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.263345 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"5ae9bbde625bf3c28413f628d1617b86956794637d6869c2f36e5552a005572b"} Jul 23 18:06:35.264703 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.264681 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5n6tt" event={"ID":"a56fdb46-cda2-4a61-afbd-5b46df9a57c3","Type":"ContainerStarted","Data":"2a101513271f56c00a7bef0d741e0e5a3d85cd7fa91f881423bb4bb4e68fe267"} Jul 23 18:06:35.272563 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.272510 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-128.ec2.internal" podStartSLOduration=2.27249564 podStartE2EDuration="2.27249564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:35.272476402 +0000 UTC m=+4.969597477" watchObservedRunningTime="2026-07-23 18:06:35.27249564 +0000 UTC m=+4.969616714" Jul 23 18:06:35.533325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.533083 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:35.533325 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.533244 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:35.533325 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.533318 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret podName:1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:37.533298617 +0000 UTC m=+7.230419668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret") pod "global-pull-secret-syncer-tlbgv" (UID: "1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:35.634144 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.634105 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:35.634328 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.634277 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:35.634419 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.634343 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs podName:f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:37.634324155 +0000 UTC m=+7.331445211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs") pod "network-metrics-daemon-ldl42" (UID: "f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:35.734782 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.734745 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:35.734989 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.734959 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:06:35.735031 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.734993 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:06:35.735031 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.735005 2555 projected.go:196] Error preparing data for projected volume kube-api-access-6svd9 for pod openshift-network-diagnostics/network-check-target-mlbqg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:35.735101 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:35.735067 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9 podName:53440927-f60b-4282-a510-d08f54e11f45 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:37.73504809 +0000 UTC m=+7.432169159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6svd9" (UniqueName: "kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9") pod "network-check-target-mlbqg" (UID: "53440927-f60b-4282-a510-d08f54e11f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:35.961743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.961616 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:01:33 +0000 UTC" deadline="2028-02-08 16:18:48.941903385 +0000 UTC" Jul 23 18:06:35.961743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:35.961684 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13558h12m12.980225595s" Jul 23 18:06:36.049264 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:36.048894 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 18:06:36.288601 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:36.288504 2555 generic.go:356] "Generic (PLEG): container finished" podID="f0869f205358e1c1285fb845001c950b" containerID="c4137844e28399b69f94cfaf43157358cfd65eb87965e3147929b98aaee35e9c" exitCode=0 Jul 23 18:06:36.289654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:36.289609 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" event={"ID":"f0869f205358e1c1285fb845001c950b","Type":"ContainerDied","Data":"c4137844e28399b69f94cfaf43157358cfd65eb87965e3147929b98aaee35e9c"} Jul 23 18:06:37.241848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:37.241806 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:37.242320 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.241946 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:37.242444 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:37.242361 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:37.242516 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.242480 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:37.242579 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:37.242536 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:37.242632 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.242611 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:37.301345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:37.301304 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" event={"ID":"f0869f205358e1c1285fb845001c950b","Type":"ContainerStarted","Data":"8c23cfc94b2c7ac835975fefa6f62f9e57b26464af682f8871b8436bfeceb7df"} Jul 23 18:06:37.550961 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:37.550327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:37.550961 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.550492 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:37.550961 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.550554 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret podName:1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:41.550537074 +0000 UTC m=+11.247658127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret") pod "global-pull-secret-syncer-tlbgv" (UID: "1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:37.651443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:37.651357 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:37.651624 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.651569 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:37.651687 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.651640 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs podName:f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:41.651619731 +0000 UTC m=+11.348740807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs") pod "network-metrics-daemon-ldl42" (UID: "f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:37.752334 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:37.752268 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:37.752539 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.752527 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:06:37.752612 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.752546 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:06:37.752612 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.752560 2555 projected.go:196] Error preparing data for projected volume kube-api-access-6svd9 for pod openshift-network-diagnostics/network-check-target-mlbqg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:37.752720 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:37.752627 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9 podName:53440927-f60b-4282-a510-d08f54e11f45 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:41.752607297 +0000 UTC m=+11.449728352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6svd9" (UniqueName: "kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9") pod "network-check-target-mlbqg" (UID: "53440927-f60b-4282-a510-d08f54e11f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:39.242086 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:39.242017 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:39.242086 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:39.242045 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:39.242517 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:39.242030 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:39.242517 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:39.242159 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:39.242517 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:39.242237 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:39.242517 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:39.242321 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:41.241175 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:41.241137 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:41.241627 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.241288 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:41.241627 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:41.241506 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:41.241627 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.241612 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:41.241729 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:41.241665 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:41.241762 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.241745 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:41.585132 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:41.584995 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:41.585309 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.585150 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:41.585309 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.585240 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret podName:1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:49.585210198 +0000 UTC m=+19.282331320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret") pod "global-pull-secret-syncer-tlbgv" (UID: "1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:41.686612 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:41.686015 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:41.686612 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.686183 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:41.686612 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.686253 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs podName:f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:49.686232705 +0000 UTC m=+19.383353776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs") pod "network-metrics-daemon-ldl42" (UID: "f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:41.786731 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:41.786644 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:41.786920 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.786842 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:06:41.786920 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.786866 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:06:41.786920 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.786880 2555 projected.go:196] Error preparing data for projected volume kube-api-access-6svd9 for pod openshift-network-diagnostics/network-check-target-mlbqg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:41.787073 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:41.786925 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9 podName:53440927-f60b-4282-a510-d08f54e11f45 nodeName:}" failed. No retries permitted until 2026-07-23 18:06:49.786912847 +0000 UTC m=+19.484033899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6svd9" (UniqueName: "kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9") pod "network-check-target-mlbqg" (UID: "53440927-f60b-4282-a510-d08f54e11f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:43.242228 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:43.241883 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:43.242228 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:43.242039 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:43.242712 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:43.242432 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:43.242712 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:43.242544 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:43.242712 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:43.242613 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:43.242712 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:43.242683 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:45.241326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:45.241189 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:45.241326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:45.241223 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:45.241326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:45.241226 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:45.241857 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:45.241329 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:45.241857 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:45.241390 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:45.241857 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:45.241483 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:47.241979 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:47.241938 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:47.242415 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:47.241949 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:47.242415 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:47.241958 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:47.242415 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:47.242188 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:47.242415 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:47.242071 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:47.242415 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:47.242256 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:49.241290 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:49.241251 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:49.241771 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:49.241251 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:49.241771 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.241406 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:49.241771 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.241449 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:49.241771 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:49.241251 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:49.241771 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.241511 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:49.643774 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:49.643730 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:49.643950 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.643860 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:49.643950 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.643928 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret podName:1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:05.643914049 +0000 UTC m=+35.341035101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret") pod "global-pull-secret-syncer-tlbgv" (UID: "1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:06:49.744876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:49.744794 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:49.745078 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.744985 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:49.745078 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.745065 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs podName:f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:05.745044779 +0000 UTC m=+35.442165836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs") pod "network-metrics-daemon-ldl42" (UID: "f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:06:49.846053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:49.846008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:49.846228 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.846185 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:06:49.846228 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.846211 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:06:49.846228 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.846224 2555 projected.go:196] Error preparing data for projected volume kube-api-access-6svd9 for pod openshift-network-diagnostics/network-check-target-mlbqg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:49.846395 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:49.846287 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9 podName:53440927-f60b-4282-a510-d08f54e11f45 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:05.84626888 +0000 UTC m=+35.543389949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6svd9" (UniqueName: "kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9") pod "network-check-target-mlbqg" (UID: "53440927-f60b-4282-a510-d08f54e11f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:06:51.241672 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:51.241633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:51.242125 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:51.241632 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:51.242125 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:51.241769 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:51.242125 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:51.241633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:51.242125 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:51.241843 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:51.242125 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:51.241927 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:53.241530 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:53.241488 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:53.242056 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:53.241508 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:53.242056 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:53.241636 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:53.242056 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:53.241701 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:53.242056 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:53.241489 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:53.242056 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:53.241827 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:54.337840 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.337477 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-srsjc" event={"ID":"3491db21-f1c1-4f95-8eb5-95938c6657e6","Type":"ContainerStarted","Data":"6cda9d3ea98e8a6fcae89a260ac335f18d3fc52936f757b6c32cc72bfb9b3467"} Jul 23 18:06:54.339618 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.339598 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:06:54.339913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.339894 2555 generic.go:356] "Generic (PLEG): container finished" podID="ab021d97-ecab-41fd-8d6b-5e01f0963e4f" containerID="6b49c944fe40102b61903505cc8394697ef854984c457e4adfeb056432bc7df1" exitCode=1 Jul 23 18:06:54.339973 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.339958 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"daa1de80ed3c5ac59b63602097bdf8efee3984a1f3ad249e1cc7a9699b1f3907"} Jul 23 18:06:54.340024 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.340012 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"2593cffe67d2549c5de9b59de1530a68aa94ec645bb01acf26418f8fbd686377"} Jul 23 18:06:54.340068 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.340047 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerDied","Data":"6b49c944fe40102b61903505cc8394697ef854984c457e4adfeb056432bc7df1"} Jul 23 18:06:54.340103 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.340076 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"79a539963a7ec2a97bf5f86d2185d08d44a507a1d96310fbdd9f76adeaee7ba9"} Jul 23 18:06:54.341258 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.341226 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5n6tt" event={"ID":"a56fdb46-cda2-4a61-afbd-5b46df9a57c3","Type":"ContainerStarted","Data":"208b6a844e6d42d1f851543f155d227324ed6e8b943256313be46f94052a506f"} Jul 23 18:06:54.342578 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.342554 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2tj8f" event={"ID":"7e3d2607-3f4d-4eac-a069-ca8ae834bc85","Type":"ContainerStarted","Data":"6c12b2bd5264b725aa50dbec0578dd5c6ec5dd453f974c609f2d1fe6137e4964"} Jul 23 18:06:54.343816 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.343795 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wss72" event={"ID":"1e71b11c-141b-467a-9e9d-deb696739f55","Type":"ContainerStarted","Data":"fe1826d77d171c058eadd3da8913457f496b5bbd1bfd3bb904b1db002cbcd94d"} Jul 23 18:06:54.346657 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.346632 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" event={"ID":"80f84c0a-f11e-43de-bd24-d9d5abba846d","Type":"ContainerStarted","Data":"59efd3ce4bcbaaa28c3356e4837513aa30f0e5cf29ce0ef338a0ca0fe9d78c5f"} Jul 23 18:06:54.348147 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.348115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5sfpq" event={"ID":"28429fa2-bd08-431a-9c1f-7d191d397d01","Type":"ContainerStarted","Data":"9bed2aba08bd08b02639ded4213e3cc48210ab311ba043ddbe8a71543efed3de"} Jul 23 18:06:54.349532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.349512 2555 generic.go:356] "Generic (PLEG): container finished" podID="ad9d3a05-d68e-4fb4-9dff-6333cb8420ec" containerID="30bf1d8e17f9971cb17d9ea283fbafc1cd509e1af2157508041f0acaf25c7acd" exitCode=0 Jul 23 18:06:54.349611 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.349541 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerDied","Data":"30bf1d8e17f9971cb17d9ea283fbafc1cd509e1af2157508041f0acaf25c7acd"} Jul 23 18:06:54.359436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.359390 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-srsjc" podStartSLOduration=3.849460335 podStartE2EDuration="22.359349457s" totalImagesPullingTime="18.509889122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:54.359069795 +0000 UTC m=+24.056190897" watchObservedRunningTime="2026-07-23 18:06:54.359349457 +0000 UTC m=+24.056470530" Jul 23 18:06:54.359617 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.359592 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-128.ec2.internal" podStartSLOduration=21.35955371 podStartE2EDuration="21.35955371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:37.325771243 +0000 UTC m=+7.022892318" watchObservedRunningTime="2026-07-23 18:06:54.35955371 +0000 UTC m=+24.056674785" Jul 23 18:06:54.410826 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.410772 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5n6tt" podStartSLOduration=8.488549309 podStartE2EDuration="22.410756344s" totalImagesPullingTime="13.922207035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:54.410195178 +0000 UTC m=+24.107316277" watchObservedRunningTime="2026-07-23 18:06:54.410756344 +0000 UTC m=+24.107877418" Jul 23 18:06:54.457026 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.456970 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2tj8f" podStartSLOduration=3.979651626 podStartE2EDuration="22.456952121s" totalImagesPullingTime="18.477300495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:54.431402947 +0000 UTC m=+24.128524022" watchObservedRunningTime="2026-07-23 18:06:54.456952121 +0000 UTC m=+24.154073265" Jul 23 18:06:54.474378 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.474254 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wss72" podStartSLOduration=3.970002231 podStartE2EDuration="22.474237468s" totalImagesPullingTime="18.504235237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:54.47347367 +0000 UTC m=+24.170594763" watchObservedRunningTime="2026-07-23 18:06:54.474237468 +0000 UTC m=+24.171358543" Jul 23 18:06:54.474521 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:54.474452 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5sfpq" podStartSLOduration=3.938899734 podStartE2EDuration="22.474445446s" totalImagesPullingTime="18.535545712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:54.456800497 +0000 UTC m=+24.153921602" watchObservedRunningTime="2026-07-23 18:06:54.474445446 +0000 UTC m=+24.171566523" Jul 23 18:06:55.241406 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.241331 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:55.241539 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.241353 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:55.241539 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:55.241501 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:55.241647 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.241353 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:55.241647 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:55.241572 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:55.241751 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:55.241657 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:55.254618 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.254588 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 18:06:55.353609 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.353524 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" event={"ID":"80f84c0a-f11e-43de-bd24-d9d5abba846d","Type":"ContainerStarted","Data":"5121ea70488b50ccffa7de77f6722c813a31cf624c0720cd9a16ed778a90b7fc"} Jul 23 18:06:55.356091 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.356070 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:06:55.356463 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.356436 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"65294c2e8c13321bb16579bb9ada9832653e1b4bfb52b456b585b6ecd3b7e30b"} Jul 23 18:06:55.356544 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.356504 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"42bdc1d4707489caa854aafc4c168347fe9e6b026063b79dc7f69304c3643373"} Jul 23 18:06:55.357791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.357764 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8jzs5" event={"ID":"3e36dcf4-2a4f-43af-8358-851a13af2f03","Type":"ContainerStarted","Data":"1bbc7d10a9dca3bb6d6c1f414c15837fd7172cced49573e1cc5278f037be112e"} Jul 23 18:06:55.378020 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.377957 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8jzs5" podStartSLOduration=4.8704658389999995 podStartE2EDuration="23.377939477s" totalImagesPullingTime="18.507473638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:55.377434889 +0000 UTC m=+25.074555992" watchObservedRunningTime="2026-07-23 18:06:55.377939477 +0000 UTC m=+25.075060550" Jul 23 18:06:55.980332 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.980216 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T18:06:55.254612625Z","UUID":"002ab571-25b0-4395-b8ab-277480459958","Handler":null,"Name":"","Endpoint":""} Jul 23 18:06:55.982005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.981981 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 18:06:55.982158 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:55.982028 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 18:06:56.362764 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:56.362726 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" event={"ID":"80f84c0a-f11e-43de-bd24-d9d5abba846d","Type":"ContainerStarted","Data":"876f6b1e1e9413c738d9212bf41000f48a3c975ed876cf884357ecc00056d232"} Jul 23 18:06:56.405599 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:56.405539 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tmdh4" podStartSLOduration=3.690167931 podStartE2EDuration="24.405517706s" totalImagesPullingTime="20.715349775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:06:56.405422826 +0000 UTC m=+26.102543935" watchObservedRunningTime="2026-07-23 18:06:56.405517706 +0000 UTC m=+26.102638781" Jul 23 18:06:57.241634 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:57.241594 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:57.241847 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:57.241601 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:57.241847 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:57.241755 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:57.241847 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:57.241601 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:57.241847 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:57.241812 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:57.242056 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:57.241892 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:57.367347 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:57.367316 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:06:57.367781 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:57.367659 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"57f051cf431d6f4c0aa9b5080bcb5afb86da76cb253716d79036d6efb85ffff3"} Jul 23 18:06:58.783079 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:58.782816 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:58.783558 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:58.783531 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:59.241851 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.241598 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:06:59.241980 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.241640 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:06:59.241980 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:59.241883 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:06:59.241980 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.241644 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:06:59.241980 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:59.241967 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:06:59.242106 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:06:59.242047 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:06:59.374753 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.374720 2555 generic.go:356] "Generic (PLEG): container finished" podID="ad9d3a05-d68e-4fb4-9dff-6333cb8420ec" containerID="51463fff11048b8dc54c67aaebdc838d3250f23547d2bc0cd91dc3a05d0c7084" exitCode=0 Jul 23 18:06:59.374928 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.374797 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerDied","Data":"51463fff11048b8dc54c67aaebdc838d3250f23547d2bc0cd91dc3a05d0c7084"} Jul 23 18:06:59.378497 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.378474 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:06:59.378895 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.378870 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"9c11b205ccb2590ec6482b8d4e0d7a156d7608656fa16d26a516ff5fb83dadbb"} Jul 23 18:06:59.379043 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.379028 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:06:59.379257 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.379244 2555 scope.go:117] "RemoveContainer" containerID="6b49c944fe40102b61903505cc8394697ef854984c457e4adfeb056432bc7df1" Jul 23 18:06:59.379491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:06:59.379477 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5n6tt" Jul 23 18:07:00.384682 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.384604 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:07:00.385471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.384967 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" event={"ID":"ab021d97-ecab-41fd-8d6b-5e01f0963e4f","Type":"ContainerStarted","Data":"8c076af4de82e55531fef9fc6f88845220f27d1c85647edafaa1c74062fb6ef6"} Jul 23 18:07:00.385471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.385129 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:07:00.385471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.385201 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:07:00.385471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.385252 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:07:00.392608 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.392565 2555 generic.go:356] "Generic (PLEG): container finished" podID="ad9d3a05-d68e-4fb4-9dff-6333cb8420ec" containerID="5f651150aaeec02f43b57f218eec6c2944f9c93f7d52bff534e2391530a38a7e" exitCode=0 Jul 23 18:07:00.392741 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.392614 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerDied","Data":"5f651150aaeec02f43b57f218eec6c2944f9c93f7d52bff534e2391530a38a7e"} Jul 23 18:07:00.402133 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.402108 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:07:00.402331 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.402313 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:07:00.432604 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:00.432544 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" podStartSLOduration=9.647049966 podStartE2EDuration="28.432528852s" totalImagesPullingTime="18.785478886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:00.43223735 +0000 UTC m=+30.129358425" watchObservedRunningTime="2026-07-23 18:07:00.432528852 +0000 UTC m=+30.129649925" Jul 23 18:07:01.241285 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.241245 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:01.241505 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.241245 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:01.241505 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.241245 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:01.241505 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:01.241407 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:07:01.241505 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:01.241456 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:07:01.241702 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:01.241519 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:07:01.398748 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.398651 2555 generic.go:356] "Generic (PLEG): container finished" podID="ad9d3a05-d68e-4fb4-9dff-6333cb8420ec" containerID="ca63c2a83249075493521948ecbec429e633d3bca94f3c740c67182c9d5d7840" exitCode=0 Jul 23 18:07:01.398748 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.398687 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerDied","Data":"ca63c2a83249075493521948ecbec429e633d3bca94f3c740c67182c9d5d7840"} Jul 23 18:07:01.413982 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.413949 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldl42"] Jul 23 18:07:01.414106 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.414056 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:01.414187 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:01.414168 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:07:01.419115 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.419081 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tlbgv"] Jul 23 18:07:01.419289 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.419207 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:01.419356 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:01.419318 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:07:01.423120 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.423095 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mlbqg"] Jul 23 18:07:01.423225 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:01.423186 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:01.423284 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:01.423264 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:07:03.241863 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:03.241821 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:03.242281 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:03.241819 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:03.242281 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:03.241943 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:07:03.242281 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:03.241975 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:03.242281 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:03.242097 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:07:03.242281 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:03.242201 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:07:05.241798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:05.241760 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:05.241798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:05.241782 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:05.241798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:05.241799 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:05.242549 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.241894 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mlbqg" podUID="53440927-f60b-4282-a510-d08f54e11f45" Jul 23 18:07:05.242549 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.242023 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldl42" podUID="f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1" Jul 23 18:07:05.242549 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.242112 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbgv" podUID="1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4" Jul 23 18:07:05.668800 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:05.668764 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:05.668990 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.668872 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 18:07:05.668990 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.668937 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret podName:1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:37.668919186 +0000 UTC m=+67.366040246 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret") pod "global-pull-secret-syncer-tlbgv" (UID: "1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4") : object "kube-system"/"original-pull-secret" not registered Jul 23 18:07:05.769608 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:05.769577 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:05.769759 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.769677 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:07:05.769759 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.769729 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs podName:f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:37.769713951 +0000 UTC m=+67.466835003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs") pod "network-metrics-daemon-ldl42" (UID: "f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 18:07:05.870719 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:05.870692 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:05.870865 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.870849 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 18:07:05.870905 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.870868 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 18:07:05.870905 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.870877 2555 projected.go:196] Error preparing data for projected volume kube-api-access-6svd9 for pod openshift-network-diagnostics/network-check-target-mlbqg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:07:05.870967 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:05.870925 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9 podName:53440927-f60b-4282-a510-d08f54e11f45 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:37.870912113 +0000 UTC m=+67.568033164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6svd9" (UniqueName: "kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9") pod "network-check-target-mlbqg" (UID: "53440927-f60b-4282-a510-d08f54e11f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 18:07:06.413763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.413728 2555 generic.go:356] "Generic (PLEG): container finished" podID="ad9d3a05-d68e-4fb4-9dff-6333cb8420ec" containerID="cc77426fa7482e4c7c16a2bdd55ccf5e0c7bdbe1e8b16ea98f5bd6f0331e5d1a" exitCode=0 Jul 23 18:07:06.414449 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.413777 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerDied","Data":"cc77426fa7482e4c7c16a2bdd55ccf5e0c7bdbe1e8b16ea98f5bd6f0331e5d1a"} Jul 23 18:07:06.664808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.664776 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-128.ec2.internal" event="NodeReady" Jul 23 18:07:06.664974 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.664898 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 18:07:06.740753 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.740716 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59988fc767-f6sjl"] Jul 23 18:07:06.749543 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.749512 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.754019 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.753990 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 18:07:06.754130 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.754080 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 18:07:06.754130 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.754080 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c6cqn\"" Jul 23 18:07:06.757238 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.757196 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9l82s"] Jul 23 18:07:06.757738 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.757720 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 18:07:06.773313 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.773289 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.773715 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.773695 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bpdcc"] Jul 23 18:07:06.776283 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.776260 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 18:07:06.776539 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.776520 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kfp27\"" Jul 23 18:07:06.776735 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.776717 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 18:07:06.779224 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.779175 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 18:07:06.785439 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.785418 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59988fc767-f6sjl"] Jul 23 18:07:06.785538 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.785504 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:06.787471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.787448 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9l82s"] Jul 23 18:07:06.793967 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.793952 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 18:07:06.795223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.795205 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mtn54\"" Jul 23 18:07:06.795295 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.795279 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 18:07:06.796206 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.796188 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bpdcc"] Jul 23 18:07:06.797330 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.797311 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 18:07:06.876979 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.876945 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-image-registry-private-configuration\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.876979 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.876982 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h48nl\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-kube-api-access-h48nl\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.877215 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877004 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-trusted-ca\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.877215 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877091 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqp6l\" (UniqueName: \"kubernetes.io/projected/1ffe51b0-c8c4-470c-8391-13e33dfca268-kube-api-access-nqp6l\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.877215 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877109 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5q94\" (UniqueName: \"kubernetes.io/projected/f2aeac23-dcb6-43ba-b1e8-9646614ed07a-kube-api-access-r5q94\") pod \"ingress-canary-bpdcc\" (UID: \"f2aeac23-dcb6-43ba-b1e8-9646614ed07a\") " pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:06.877215 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877125 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.877215 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877191 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-bound-sa-token\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.877412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877222 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.877412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877260 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-ca-trust-extracted\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.877412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-installation-pull-secrets\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.877412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ffe51b0-c8c4-470c-8391-13e33dfca268-config-volume\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.877412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877325 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ffe51b0-c8c4-470c-8391-13e33dfca268-tmp-dir\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.877412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877341 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2aeac23-dcb6-43ba-b1e8-9646614ed07a-cert\") pod \"ingress-canary-bpdcc\" (UID: \"f2aeac23-dcb6-43ba-b1e8-9646614ed07a\") " pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:06.877412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.877384 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-certificates\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.978525 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ffe51b0-c8c4-470c-8391-13e33dfca268-config-volume\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.978525 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978527 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ffe51b0-c8c4-470c-8391-13e33dfca268-tmp-dir\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.978525 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978544 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2aeac23-dcb6-43ba-b1e8-9646614ed07a-cert\") pod \"ingress-canary-bpdcc\" (UID: \"f2aeac23-dcb6-43ba-b1e8-9646614ed07a\") " pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:06.978832 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978561 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-certificates\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.978832 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978596 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-image-registry-private-configuration\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.978832 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h48nl\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-kube-api-access-h48nl\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.978832 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978644 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-trusted-ca\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.978832 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978669 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqp6l\" (UniqueName: \"kubernetes.io/projected/1ffe51b0-c8c4-470c-8391-13e33dfca268-kube-api-access-nqp6l\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.979042 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5q94\" (UniqueName: \"kubernetes.io/projected/f2aeac23-dcb6-43ba-b1e8-9646614ed07a-kube-api-access-r5q94\") pod \"ingress-canary-bpdcc\" (UID: \"f2aeac23-dcb6-43ba-b1e8-9646614ed07a\") " pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:06.979042 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978944 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ffe51b0-c8c4-470c-8391-13e33dfca268-tmp-dir\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.979042 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.978971 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.979042 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.979033 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-bound-sa-token\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.979206 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.979069 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.979206 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.979146 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-ca-trust-extracted\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.979206 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.979176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-installation-pull-secrets\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.979396 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.979346 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-certificates\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.979396 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:06.979390 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:07:06.979520 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:06.979407 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59988fc767-f6sjl: secret "image-registry-tls" not found Jul 23 18:07:06.979520 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:06.979457 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls podName:d70b858b-5f3b-4b5e-b55e-b03f5e7691c2 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:07.479439713 +0000 UTC m=+37.176560765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls") pod "image-registry-59988fc767-f6sjl" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2") : secret "image-registry-tls" not found Jul 23 18:07:06.979520 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:06.979465 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:07:06.979666 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:06.979539 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls podName:1ffe51b0-c8c4-470c-8391-13e33dfca268 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:07.479520101 +0000 UTC m=+37.176641372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls") pod "dns-default-9l82s" (UID: "1ffe51b0-c8c4-470c-8391-13e33dfca268") : secret "dns-default-metrics-tls" not found Jul 23 18:07:06.979731 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.979675 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-ca-trust-extracted\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.979872 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.979851 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ffe51b0-c8c4-470c-8391-13e33dfca268-config-volume\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:06.980218 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.980198 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-trusted-ca\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.983147 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.983128 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-installation-pull-secrets\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.983212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.983182 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2aeac23-dcb6-43ba-b1e8-9646614ed07a-cert\") pod \"ingress-canary-bpdcc\" (UID: \"f2aeac23-dcb6-43ba-b1e8-9646614ed07a\") " pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:06.983478 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.983461 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-image-registry-private-configuration\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.994813 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.994787 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5q94\" (UniqueName: \"kubernetes.io/projected/f2aeac23-dcb6-43ba-b1e8-9646614ed07a-kube-api-access-r5q94\") pod \"ingress-canary-bpdcc\" (UID: \"f2aeac23-dcb6-43ba-b1e8-9646614ed07a\") " pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:06.996087 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.996064 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-bound-sa-token\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.996177 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.996126 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h48nl\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-kube-api-access-h48nl\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:06.997012 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:06.996990 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqp6l\" (UniqueName: \"kubernetes.io/projected/1ffe51b0-c8c4-470c-8391-13e33dfca268-kube-api-access-nqp6l\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:07.093593 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.093556 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bpdcc" Jul 23 18:07:07.241920 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.241736 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:07.242081 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.241765 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:07.242159 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.241769 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:07.246094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.246073 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 18:07:07.246273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.246074 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 18:07:07.246273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.246074 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bbklj\"" Jul 23 18:07:07.247022 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.247001 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 18:07:07.247022 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.247017 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t929l\"" Jul 23 18:07:07.247155 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.247022 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 18:07:07.310700 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.310670 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bpdcc"] Jul 23 18:07:07.315445 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:07.315416 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2aeac23_dcb6_43ba_b1e8_9646614ed07a.slice/crio-f7cb699d4c08bc70534db5b7e46c51af92ac3b9de6b9360d91cb96d47262d195 WatchSource:0}: Error finding container f7cb699d4c08bc70534db5b7e46c51af92ac3b9de6b9360d91cb96d47262d195: Status 404 returned error can't find the container with id f7cb699d4c08bc70534db5b7e46c51af92ac3b9de6b9360d91cb96d47262d195 Jul 23 18:07:07.418691 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.418572 2555 generic.go:356] "Generic (PLEG): container finished" podID="ad9d3a05-d68e-4fb4-9dff-6333cb8420ec" containerID="34416218b7614230df447981dce3dbcd1ec55e2b65b39a7b2190d1f168dd50e3" exitCode=0 Jul 23 18:07:07.418691 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.418651 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerDied","Data":"34416218b7614230df447981dce3dbcd1ec55e2b65b39a7b2190d1f168dd50e3"} Jul 23 18:07:07.419791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.419761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bpdcc" event={"ID":"f2aeac23-dcb6-43ba-b1e8-9646614ed07a","Type":"ContainerStarted","Data":"f7cb699d4c08bc70534db5b7e46c51af92ac3b9de6b9360d91cb96d47262d195"} Jul 23 18:07:07.483535 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.483502 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:07.483680 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:07.483550 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:07.483680 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:07.483644 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:07:07.483680 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:07.483661 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:07:07.483680 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:07.483682 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59988fc767-f6sjl: secret "image-registry-tls" not found Jul 23 18:07:07.483808 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:07.483695 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls podName:1ffe51b0-c8c4-470c-8391-13e33dfca268 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:08.483683083 +0000 UTC m=+38.180804135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls") pod "dns-default-9l82s" (UID: "1ffe51b0-c8c4-470c-8391-13e33dfca268") : secret "dns-default-metrics-tls" not found Jul 23 18:07:07.483808 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:07.483739 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls podName:d70b858b-5f3b-4b5e-b55e-b03f5e7691c2 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:08.483724634 +0000 UTC m=+38.180845685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls") pod "image-registry-59988fc767-f6sjl" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2") : secret "image-registry-tls" not found Jul 23 18:07:08.426248 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:08.426209 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n28cd" event={"ID":"ad9d3a05-d68e-4fb4-9dff-6333cb8420ec","Type":"ContainerStarted","Data":"b7a40da6a5d02f25cc05673f4db499de26295c57ba122c7e5cb4632581c8b29d"} Jul 23 18:07:08.462616 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:08.462562 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n28cd" podStartSLOduration=7.401422519 podStartE2EDuration="36.462542887s" totalImagesPullingTime="29.061120368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:08.46179702 +0000 UTC m=+38.158918140" watchObservedRunningTime="2026-07-23 18:07:08.462542887 +0000 UTC m=+38.159663960" Jul 23 18:07:08.492356 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:08.492316 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:08.492557 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:08.492388 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:08.492557 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:08.492496 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:07:08.492557 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:08.492502 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:07:08.492557 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:08.492523 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59988fc767-f6sjl: secret "image-registry-tls" not found Jul 23 18:07:08.492557 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:08.492550 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls podName:1ffe51b0-c8c4-470c-8391-13e33dfca268 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:10.492535819 +0000 UTC m=+40.189656871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls") pod "dns-default-9l82s" (UID: "1ffe51b0-c8c4-470c-8391-13e33dfca268") : secret "dns-default-metrics-tls" not found Jul 23 18:07:08.492759 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:08.492587 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls podName:d70b858b-5f3b-4b5e-b55e-b03f5e7691c2 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:10.492563168 +0000 UTC m=+40.189684235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls") pod "image-registry-59988fc767-f6sjl" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2") : secret "image-registry-tls" not found Jul 23 18:07:08.799471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:08.799356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:08.905136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:08.905104 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:10.289413 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:10.289219 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:10.355409 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:10.355359 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:10.432691 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:10.432658 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bpdcc" event={"ID":"f2aeac23-dcb6-43ba-b1e8-9646614ed07a","Type":"ContainerStarted","Data":"03ee1e925f370479bc3747fca93e0f0fe6cf0f5c1f9cc30f65e36ff083cd6bd5"} Jul 23 18:07:10.456328 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:10.456264 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bpdcc" podStartSLOduration=2.079213701 podStartE2EDuration="4.456246844s" totalImagesPullingTime="2.377033143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:10.45468406 +0000 UTC m=+40.151805135" watchObservedRunningTime="2026-07-23 18:07:10.456246844 +0000 UTC m=+40.153367918" Jul 23 18:07:10.507384 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:10.507318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:10.507550 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:10.507442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:10.507550 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:10.507489 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:07:10.507663 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:10.507565 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls podName:1ffe51b0-c8c4-470c-8391-13e33dfca268 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:14.507548176 +0000 UTC m=+44.204669248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls") pod "dns-default-9l82s" (UID: "1ffe51b0-c8c4-470c-8391-13e33dfca268") : secret "dns-default-metrics-tls" not found Jul 23 18:07:10.507663 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:10.507576 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:07:10.507663 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:10.507596 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59988fc767-f6sjl: secret "image-registry-tls" not found Jul 23 18:07:10.507663 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:10.507655 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls podName:d70b858b-5f3b-4b5e-b55e-b03f5e7691c2 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:14.507638642 +0000 UTC m=+44.204759698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls") pod "image-registry-59988fc767-f6sjl" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2") : secret "image-registry-tls" not found Jul 23 18:07:11.978787 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:11.978757 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:12.135025 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:12.134993 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:14.539381 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.539323 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:14.539815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.539400 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:14.539815 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:14.539485 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:07:14.539815 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:14.539507 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59988fc767-f6sjl: secret "image-registry-tls" not found Jul 23 18:07:14.539815 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:14.539547 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:07:14.539815 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:14.539580 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls podName:d70b858b-5f3b-4b5e-b55e-b03f5e7691c2 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:22.539560864 +0000 UTC m=+52.236681947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls") pod "image-registry-59988fc767-f6sjl" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2") : secret "image-registry-tls" not found Jul 23 18:07:14.539815 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:14.539594 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls podName:1ffe51b0-c8c4-470c-8391-13e33dfca268 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:22.539588174 +0000 UTC m=+52.236709226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls") pod "dns-default-9l82s" (UID: "1ffe51b0-c8c4-470c-8391-13e33dfca268") : secret "dns-default-metrics-tls" not found Jul 23 18:07:14.699437 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.699408 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj"] Jul 23 18:07:14.729552 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.729520 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:14.735410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.735354 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 18:07:14.735410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.735398 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 23 18:07:14.735632 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.735407 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-775hg\"" Jul 23 18:07:14.736929 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.736148 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 18:07:14.736929 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.736164 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 18:07:14.736929 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.736653 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj"] Jul 23 18:07:14.842067 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.842029 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8e35d752-ec48-4a29-bc46-1b1dd93b38e3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5949c78698-b9dgj\" (UID: \"8e35d752-ec48-4a29-bc46-1b1dd93b38e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:14.842067 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.842073 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ltqt\" (UniqueName: \"kubernetes.io/projected/8e35d752-ec48-4a29-bc46-1b1dd93b38e3-kube-api-access-9ltqt\") pod \"managed-serviceaccount-addon-agent-5949c78698-b9dgj\" (UID: \"8e35d752-ec48-4a29-bc46-1b1dd93b38e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:14.942763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.942730 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8e35d752-ec48-4a29-bc46-1b1dd93b38e3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5949c78698-b9dgj\" (UID: \"8e35d752-ec48-4a29-bc46-1b1dd93b38e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:14.942877 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.942772 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ltqt\" (UniqueName: \"kubernetes.io/projected/8e35d752-ec48-4a29-bc46-1b1dd93b38e3-kube-api-access-9ltqt\") pod \"managed-serviceaccount-addon-agent-5949c78698-b9dgj\" (UID: \"8e35d752-ec48-4a29-bc46-1b1dd93b38e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:14.946959 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.946928 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8e35d752-ec48-4a29-bc46-1b1dd93b38e3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5949c78698-b9dgj\" (UID: \"8e35d752-ec48-4a29-bc46-1b1dd93b38e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:14.959350 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:14.959309 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ltqt\" (UniqueName: \"kubernetes.io/projected/8e35d752-ec48-4a29-bc46-1b1dd93b38e3-kube-api-access-9ltqt\") pod \"managed-serviceaccount-addon-agent-5949c78698-b9dgj\" (UID: \"8e35d752-ec48-4a29-bc46-1b1dd93b38e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:15.058652 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:15.058616 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" Jul 23 18:07:15.217406 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:15.217354 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj"] Jul 23 18:07:15.220396 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:15.220349 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e35d752_ec48_4a29_bc46_1b1dd93b38e3.slice/crio-34275eb48ac0260c1096981c8e1676fe450678153f51ebef730bcbeff82daced WatchSource:0}: Error finding container 34275eb48ac0260c1096981c8e1676fe450678153f51ebef730bcbeff82daced: Status 404 returned error can't find the container with id 34275eb48ac0260c1096981c8e1676fe450678153f51ebef730bcbeff82daced Jul 23 18:07:15.445640 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:15.445554 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" event={"ID":"8e35d752-ec48-4a29-bc46-1b1dd93b38e3","Type":"ContainerStarted","Data":"34275eb48ac0260c1096981c8e1676fe450678153f51ebef730bcbeff82daced"} Jul 23 18:07:16.621834 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:16.621792 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:16.726210 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:16.726178 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:16.827785 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:16.827712 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:16.932666 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:16.932575 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:17.043350 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.043311 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:17.295248 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.295165 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:17.402496 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.402461 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:17.439050 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.439018 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sbwl6"] Jul 23 18:07:17.491830 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.491796 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sbwl6"] Jul 23 18:07:17.492004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.491932 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.502612 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.502587 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 18:07:17.502912 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.502897 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 18:07:17.503750 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.503728 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j5mk2\"" Jul 23 18:07:17.503879 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.503781 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 18:07:17.508410 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.508386 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 18:07:17.514635 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.514603 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:17.644566 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.644529 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:17.667545 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.667506 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.667738 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.667582 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6wmt\" (UniqueName: \"kubernetes.io/projected/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-api-access-c6wmt\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.667738 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.667608 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/720a8d7f-6625-4173-acfa-4c6df41f3519-data-volume\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.667738 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.667630 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.667738 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.667670 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/720a8d7f-6625-4173-acfa-4c6df41f3519-crio-socket\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.759797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.759759 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:17.768147 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.768096 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.768324 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.768185 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6wmt\" (UniqueName: \"kubernetes.io/projected/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-api-access-c6wmt\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.768324 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:17.768206 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:07:17.768324 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.768222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/720a8d7f-6625-4173-acfa-4c6df41f3519-data-volume\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.768324 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.768251 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.768324 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:17.768302 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:18.26827952 +0000 UTC m=+47.965400578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : configmap "kube-rbac-proxy" not found Jul 23 18:07:17.768628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.768340 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/720a8d7f-6625-4173-acfa-4c6df41f3519-crio-socket\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.768628 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:17.768358 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:07:17.768628 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:17.768454 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:18.268421908 +0000 UTC m=+47.965542961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : secret "insights-runtime-extractor-tls" not found Jul 23 18:07:17.768628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.768598 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/720a8d7f-6625-4173-acfa-4c6df41f3519-crio-socket\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.768819 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.768668 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/720a8d7f-6625-4173-acfa-4c6df41f3519-data-volume\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.780633 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.780595 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6wmt\" (UniqueName: \"kubernetes.io/projected/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-api-access-c6wmt\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:17.942471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:17.942390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:18.049781 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.049748 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:18.208604 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.208520 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:18.272942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.272902 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:18.273139 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:18.273091 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:07:18.273208 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.273141 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:18.273208 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:18.273169 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:19.273146641 +0000 UTC m=+48.970267693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : secret "insights-runtime-extractor-tls" not found Jul 23 18:07:18.273317 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:18.273211 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:07:18.273317 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:18.273264 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:19.273247836 +0000 UTC m=+48.970368894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : configmap "kube-rbac-proxy" not found Jul 23 18:07:18.332531 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.332495 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk"] Jul 23 18:07:18.347116 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.347087 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" Jul 23 18:07:18.353113 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.353084 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:18.356808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.356787 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 23 18:07:18.356922 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.356907 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 23 18:07:18.357823 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.357808 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zmg5f\"" Jul 23 18:07:18.370307 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.370274 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk"] Jul 23 18:07:18.474888 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.474789 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnzjs\" (UniqueName: \"kubernetes.io/projected/e87f696c-8bc4-4c26-87e3-0900c9f01d65-kube-api-access-vnzjs\") pod \"migrator-7cc9c7ffd7-zhsxk\" (UID: \"e87f696c-8bc4-4c26-87e3-0900c9f01d65\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" Jul 23 18:07:18.486380 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.486319 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:18.576251 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.576213 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnzjs\" (UniqueName: \"kubernetes.io/projected/e87f696c-8bc4-4c26-87e3-0900c9f01d65-kube-api-access-vnzjs\") pod \"migrator-7cc9c7ffd7-zhsxk\" (UID: \"e87f696c-8bc4-4c26-87e3-0900c9f01d65\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" Jul 23 18:07:18.590580 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.590546 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnzjs\" (UniqueName: \"kubernetes.io/projected/e87f696c-8bc4-4c26-87e3-0900c9f01d65-kube-api-access-vnzjs\") pod \"migrator-7cc9c7ffd7-zhsxk\" (UID: \"e87f696c-8bc4-4c26-87e3-0900c9f01d65\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" Jul 23 18:07:18.621817 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.621784 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:18.655794 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:18.655760 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" Jul 23 18:07:19.010205 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.010173 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk"] Jul 23 18:07:19.018025 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:19.017997 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87f696c_8bc4_4c26_87e3_0900c9f01d65.slice/crio-1e21cf17e1241fcb16bb0353577c260f7a3f36a9274440a75c88919dbcc00c19 WatchSource:0}: Error finding container 1e21cf17e1241fcb16bb0353577c260f7a3f36a9274440a75c88919dbcc00c19: Status 404 returned error can't find the container with id 1e21cf17e1241fcb16bb0353577c260f7a3f36a9274440a75c88919dbcc00c19 Jul 23 18:07:19.233277 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.233198 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2tj8f_7e3d2607-3f4d-4eac-a069-ca8ae834bc85/dns-node-resolver/0.log" Jul 23 18:07:19.233478 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.233456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:19.282236 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.282196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:19.282448 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.282261 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:19.282448 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:19.282321 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 18:07:19.282448 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:19.282408 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:21.282392933 +0000 UTC m=+50.979514004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : configmap "kube-rbac-proxy" not found Jul 23 18:07:19.282448 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:19.282407 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:07:19.282448 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:19.282437 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:21.282430483 +0000 UTC m=+50.979551535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : secret "insights-runtime-extractor-tls" not found Jul 23 18:07:19.343386 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.343335 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:19.458135 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.458096 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" event={"ID":"8e35d752-ec48-4a29-bc46-1b1dd93b38e3","Type":"ContainerStarted","Data":"7f0c25c0020e169d5394c084977a6a5a4c986e90f90a41278df8e22783fbfa06"} Jul 23 18:07:19.459908 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.459865 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" event={"ID":"e87f696c-8bc4-4c26-87e3-0900c9f01d65","Type":"ContainerStarted","Data":"1e21cf17e1241fcb16bb0353577c260f7a3f36a9274440a75c88919dbcc00c19"} Jul 23 18:07:19.577896 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.577793 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5949c78698-b9dgj" podStartSLOduration=1.941874222 podStartE2EDuration="5.577774281s" totalImagesPullingTime="3.635900059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:19.491519126 +0000 UTC m=+49.188640199" watchObservedRunningTime="2026-07-23 18:07:19.577774281 +0000 UTC m=+49.274895358" Jul 23 18:07:19.578401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.578385 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ff7zk"] Jul 23 18:07:19.596757 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.596723 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.606179 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.606153 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ff7zk"] Jul 23 18:07:19.612004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.611974 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 23 18:07:19.612142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.612019 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-42tpl\"" Jul 23 18:07:19.612142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.612085 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 23 18:07:19.612824 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.612806 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 23 18:07:19.612998 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.612986 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 23 18:07:19.670852 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.670810 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:19.787466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.787421 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqjz\" (UniqueName: \"kubernetes.io/projected/c6b18a48-e8c8-4717-a08a-03658dbe537f-kube-api-access-9hqjz\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.787635 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.787529 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c6b18a48-e8c8-4717-a08a-03658dbe537f-signing-key\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.787635 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.787599 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c6b18a48-e8c8-4717-a08a-03658dbe537f-signing-cabundle\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.814315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.814288 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:19.888304 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.888261 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqjz\" (UniqueName: \"kubernetes.io/projected/c6b18a48-e8c8-4717-a08a-03658dbe537f-kube-api-access-9hqjz\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.888505 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.888445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c6b18a48-e8c8-4717-a08a-03658dbe537f-signing-key\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.888570 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.888522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c6b18a48-e8c8-4717-a08a-03658dbe537f-signing-cabundle\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.889298 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.889267 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c6b18a48-e8c8-4717-a08a-03658dbe537f-signing-cabundle\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.891238 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.891213 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c6b18a48-e8c8-4717-a08a-03658dbe537f-signing-key\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.903007 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.902977 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqjz\" (UniqueName: \"kubernetes.io/projected/c6b18a48-e8c8-4717-a08a-03658dbe537f-kube-api-access-9hqjz\") pod \"service-ca-58598695f7-ff7zk\" (UID: \"c6b18a48-e8c8-4717-a08a-03658dbe537f\") " pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:19.905656 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:19.905621 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-ff7zk" Jul 23 18:07:20.107021 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:20.106985 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ff7zk"] Jul 23 18:07:20.110804 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:20.110771 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b18a48_e8c8_4717_a08a_03658dbe537f.slice/crio-369a547123d233b16431dee25dd2201347240515cc8cfaaebecd801fd18d7ab4 WatchSource:0}: Error finding container 369a547123d233b16431dee25dd2201347240515cc8cfaaebecd801fd18d7ab4: Status 404 returned error can't find the container with id 369a547123d233b16431dee25dd2201347240515cc8cfaaebecd801fd18d7ab4 Jul 23 18:07:20.463960 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:20.463918 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-ff7zk" event={"ID":"c6b18a48-e8c8-4717-a08a-03658dbe537f","Type":"ContainerStarted","Data":"369a547123d233b16431dee25dd2201347240515cc8cfaaebecd801fd18d7ab4"} Jul 23 18:07:20.554194 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:20.554157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:20.632924 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:20.632896 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wss72_1e71b11c-141b-467a-9e9d-deb696739f55/node-ca/0.log" Jul 23 18:07:21.232760 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:21.232676 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bpdcc_f2aeac23-dcb6-43ba-b1e8-9646614ed07a/serve-healthcheck-canary/0.log" Jul 23 18:07:21.302228 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:21.302190 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:21.302421 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:21.302253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:21.302421 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:21.302402 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:07:21.302556 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:21.302530 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:25.302507634 +0000 UTC m=+54.999628704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : secret "insights-runtime-extractor-tls" not found Jul 23 18:07:21.302843 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:21.302819 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/720a8d7f-6625-4173-acfa-4c6df41f3519-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:21.467776 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:21.467743 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" event={"ID":"e87f696c-8bc4-4c26-87e3-0900c9f01d65","Type":"ContainerStarted","Data":"05add169cf9a536f249f29fd8f252ea397dcb6b8ac6180d777efea47e3b2e081"} Jul 23 18:07:21.467938 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:21.467801 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" event={"ID":"e87f696c-8bc4-4c26-87e3-0900c9f01d65","Type":"ContainerStarted","Data":"c4c445e06d55dc45a083b85c4921ed7440672c484ff3b288ace0e22672ce41a3"} Jul 23 18:07:21.538060 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:21.537957 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zhsxk" podStartSLOduration=1.672313644 podStartE2EDuration="3.537941139s" totalImagesPullingTime="1.865627495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:21.537792059 +0000 UTC m=+51.234913134" watchObservedRunningTime="2026-07-23 18:07:21.537941139 +0000 UTC m=+51.235062212" Jul 23 18:07:22.611503 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:22.611455 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:22.611901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:22.611522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:22.611901 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:22.611633 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 18:07:22.611901 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:22.611637 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 18:07:22.611901 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:22.611729 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls podName:1ffe51b0-c8c4-470c-8391-13e33dfca268 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:38.611708172 +0000 UTC m=+68.308829226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls") pod "dns-default-9l82s" (UID: "1ffe51b0-c8c4-470c-8391-13e33dfca268") : secret "dns-default-metrics-tls" not found Jul 23 18:07:22.611901 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:22.611728 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59988fc767-f6sjl: secret "image-registry-tls" not found Jul 23 18:07:22.611901 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:22.611760 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls podName:d70b858b-5f3b-4b5e-b55e-b03f5e7691c2 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:38.611754492 +0000 UTC m=+68.308875547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls") pod "image-registry-59988fc767-f6sjl" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2") : secret "image-registry-tls" not found Jul 23 18:07:22.886082 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:22.886002 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:23.475887 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:23.475850 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-ff7zk" event={"ID":"c6b18a48-e8c8-4717-a08a-03658dbe537f","Type":"ContainerStarted","Data":"13b5b4c8a0600c6e685f04c39327b21ad372ec2fe1fc6a6c78fc3cf57dc4f3ea"} Jul 23 18:07:23.502515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:23.502463 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-ff7zk" podStartSLOduration=1.6657566780000002 podStartE2EDuration="4.502448333s" totalImagesPullingTime="2.836691655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:23.501281603 +0000 UTC m=+53.198402693" watchObservedRunningTime="2026-07-23 18:07:23.502448333 +0000 UTC m=+53.199569385" Jul 23 18:07:24.238286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:24.238252 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:24.463926 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:24.463889 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:24.553006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:24.552917 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:24.654106 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:24.654065 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:24.773827 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:24.773792 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:24.897588 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:24.897543 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:25.334655 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:25.334563 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:25.335041 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:25.334727 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 18:07:25.335041 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:25.334806 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls podName:720a8d7f-6625-4173-acfa-4c6df41f3519 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:33.334789151 +0000 UTC m=+63.031910204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sbwl6" (UID: "720a8d7f-6625-4173-acfa-4c6df41f3519") : secret "insights-runtime-extractor-tls" not found Jul 23 18:07:25.575053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:25.575014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:25.722094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:25.722065 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:26.090226 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:26.090203 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:26.314902 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:26.314867 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:26.689616 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:26.689585 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:27.346164 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:27.346125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:27.496084 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:27.496051 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:28.011073 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:28.011041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:28.154134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:28.154096 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:28.457245 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:28.457206 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:28.554990 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:28.554954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:28.975178 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:28.975147 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:29.078449 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:29.078416 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:29.296571 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:29.296483 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:29.514701 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:29.514666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:29.688742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:29.688710 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:29.787216 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:29.787186 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:30.080578 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:30.080473 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:30.223158 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:30.223124 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:30.417971 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:30.417938 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:31.105922 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:31.105890 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:31.501452 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:31.501356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:31.704611 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:31.704584 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:31.800449 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:31.800353 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:31.914199 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:31.914169 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:31.984435 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:31.984403 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:32.089458 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:32.089426 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:32.412490 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:32.412410 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qjdpg" Jul 23 18:07:33.090508 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.090461 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:33.204379 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.204330 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:33.402303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.402201 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:33.404501 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.404469 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/720a8d7f-6625-4173-acfa-4c6df41f3519-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sbwl6\" (UID: \"720a8d7f-6625-4173-acfa-4c6df41f3519\") " pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:33.508226 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.508195 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:33.705005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.704913 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j5mk2\"" Jul 23 18:07:33.708134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.708110 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:33.712184 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.712162 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sbwl6" Jul 23 18:07:33.867526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.867501 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sbwl6"] Jul 23 18:07:33.868716 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:33.868692 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720a8d7f_6625_4173_acfa_4c6df41f3519.slice/crio-653e18318047a3c064f19bbb7c8a6609597389a7b660082c8c48f786fcd13bca WatchSource:0}: Error finding container 653e18318047a3c064f19bbb7c8a6609597389a7b660082c8c48f786fcd13bca: Status 404 returned error can't find the container with id 653e18318047a3c064f19bbb7c8a6609597389a7b660082c8c48f786fcd13bca Jul 23 18:07:33.903315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:33.903271 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:34.284801 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:34.284764 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:34.510137 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:34.510098 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbwl6" event={"ID":"720a8d7f-6625-4173-acfa-4c6df41f3519","Type":"ContainerStarted","Data":"653e18318047a3c064f19bbb7c8a6609597389a7b660082c8c48f786fcd13bca"} Jul 23 18:07:34.679088 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:34.679034 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:34.889079 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:34.889007 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:37.738592 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.738549 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:37.741802 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.741775 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 18:07:37.751582 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.751547 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4-original-pull-secret\") pod \"global-pull-secret-syncer-tlbgv\" (UID: \"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4\") " pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:37.838990 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.838948 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:37.841714 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.841688 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 18:07:37.852171 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.852140 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbgv" Jul 23 18:07:37.852402 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.852383 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1-metrics-certs\") pod \"network-metrics-daemon-ldl42\" (UID: \"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1\") " pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:37.861132 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.861103 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t929l\"" Jul 23 18:07:37.869310 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.869270 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldl42" Jul 23 18:07:37.940098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.940054 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:37.943488 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.943461 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 18:07:37.953699 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.953664 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 18:07:37.964360 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:37.964055 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6svd9\" (UniqueName: \"kubernetes.io/projected/53440927-f60b-4282-a510-d08f54e11f45-kube-api-access-6svd9\") pod \"network-check-target-mlbqg\" (UID: \"53440927-f60b-4282-a510-d08f54e11f45\") " pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:38.166101 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.166069 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bbklj\"" Jul 23 18:07:38.173653 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.173620 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:38.646189 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.646143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:38.646389 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.646211 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:38.648840 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.648811 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffe51b0-c8c4-470c-8391-13e33dfca268-metrics-tls\") pod \"dns-default-9l82s\" (UID: \"1ffe51b0-c8c4-470c-8391-13e33dfca268\") " pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:38.648950 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.648875 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"image-registry-59988fc767-f6sjl\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:38.861245 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.861208 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c6cqn\"" Jul 23 18:07:38.868670 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.868638 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:38.885053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.885023 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kfp27\"" Jul 23 18:07:38.892991 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:38.892956 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:39.150148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.150084 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:39.234544 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.234453 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:39.347625 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.347548 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tlbgv"] Jul 23 18:07:39.350847 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:39.350806 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8cac9c_1c74_4cc6_9c38_3dd55ed682f4.slice/crio-54799a189267d41e08936c4979961736c1e7a97257ac3b04c8afbbb1ea48f756 WatchSource:0}: Error finding container 54799a189267d41e08936c4979961736c1e7a97257ac3b04c8afbbb1ea48f756: Status 404 returned error can't find the container with id 54799a189267d41e08936c4979961736c1e7a97257ac3b04c8afbbb1ea48f756 Jul 23 18:07:39.366885 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.366849 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mlbqg"] Jul 23 18:07:39.369727 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:39.369691 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53440927_f60b_4282_a510_d08f54e11f45.slice/crio-d6ae5e0fd56b1fa900fc8423e2dd73b1b85e150707f1b4011b2401754a8e4bf6 WatchSource:0}: Error finding container d6ae5e0fd56b1fa900fc8423e2dd73b1b85e150707f1b4011b2401754a8e4bf6: Status 404 returned error can't find the container with id d6ae5e0fd56b1fa900fc8423e2dd73b1b85e150707f1b4011b2401754a8e4bf6 Jul 23 18:07:39.384862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.384828 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldl42"] Jul 23 18:07:39.388400 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.388033 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9l82s"] Jul 23 18:07:39.388923 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:39.388820 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf25e2eab_5ed9_4e0b_93c6_9b07325fd6d1.slice/crio-556c7cf1c7f64ddaf63e7a1412ce0aafc8b82997172fbe602e24dd5de24a016d WatchSource:0}: Error finding container 556c7cf1c7f64ddaf63e7a1412ce0aafc8b82997172fbe602e24dd5de24a016d: Status 404 returned error can't find the container with id 556c7cf1c7f64ddaf63e7a1412ce0aafc8b82997172fbe602e24dd5de24a016d Jul 23 18:07:39.393876 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:39.393853 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffe51b0_c8c4_470c_8391_13e33dfca268.slice/crio-4361b74ea3e266677e289284ae870f01b19b77ecf47b9d83ef13fcf6c0b5ef2f WatchSource:0}: Error finding container 4361b74ea3e266677e289284ae870f01b19b77ecf47b9d83ef13fcf6c0b5ef2f: Status 404 returned error can't find the container with id 4361b74ea3e266677e289284ae870f01b19b77ecf47b9d83ef13fcf6c0b5ef2f Jul 23 18:07:39.419283 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.419208 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59988fc767-f6sjl"] Jul 23 18:07:39.422462 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:39.422416 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70b858b_5f3b_4b5e_b55e_b03f5e7691c2.slice/crio-b9b14d8f8808ddc525083702352f130ba91829ce07f4fe459cc28bef2d4f1dca WatchSource:0}: Error finding container b9b14d8f8808ddc525083702352f130ba91829ce07f4fe459cc28bef2d4f1dca: Status 404 returned error can't find the container with id b9b14d8f8808ddc525083702352f130ba91829ce07f4fe459cc28bef2d4f1dca Jul 23 18:07:39.425775 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.425751 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:39.528273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.528238 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbwl6" event={"ID":"720a8d7f-6625-4173-acfa-4c6df41f3519","Type":"ContainerStarted","Data":"259dd076e44cd02ea477cfce02d81006d01806da40c3e3408554ae1873e7d984"} Jul 23 18:07:39.529422 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.529397 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mlbqg" event={"ID":"53440927-f60b-4282-a510-d08f54e11f45","Type":"ContainerStarted","Data":"d6ae5e0fd56b1fa900fc8423e2dd73b1b85e150707f1b4011b2401754a8e4bf6"} Jul 23 18:07:39.530455 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.530419 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tlbgv" event={"ID":"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4","Type":"ContainerStarted","Data":"54799a189267d41e08936c4979961736c1e7a97257ac3b04c8afbbb1ea48f756"} Jul 23 18:07:39.531353 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.531325 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9l82s" event={"ID":"1ffe51b0-c8c4-470c-8391-13e33dfca268","Type":"ContainerStarted","Data":"4361b74ea3e266677e289284ae870f01b19b77ecf47b9d83ef13fcf6c0b5ef2f"} Jul 23 18:07:39.532380 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.532350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldl42" event={"ID":"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1","Type":"ContainerStarted","Data":"556c7cf1c7f64ddaf63e7a1412ce0aafc8b82997172fbe602e24dd5de24a016d"} Jul 23 18:07:39.533678 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.533660 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" event={"ID":"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2","Type":"ContainerStarted","Data":"a029b60bbca1b472a017bbc18b21d766a4d00ececc21729c3e8ab4c985c23143"} Jul 23 18:07:39.533751 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.533689 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" event={"ID":"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2","Type":"ContainerStarted","Data":"b9b14d8f8808ddc525083702352f130ba91829ce07f4fe459cc28bef2d4f1dca"} Jul 23 18:07:39.533798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.533782 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:39.559638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.559580 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" podStartSLOduration=61.559565701 podStartE2EDuration="1m1.559565701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:39.558805053 +0000 UTC m=+69.255926152" watchObservedRunningTime="2026-07-23 18:07:39.559565701 +0000 UTC m=+69.256686775" Jul 23 18:07:39.812061 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.811973 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:39.879287 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:39.879251 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:40.012931 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:40.012885 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:40.819258 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:40.819220 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:40.916942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:40.916892 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:41.284143 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:41.284056 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:41.383316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:41.383250 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:41.550864 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:41.550752 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldl42" event={"ID":"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1","Type":"ContainerStarted","Data":"aa730a728b0f1f6354da45eef8a499cd6c87ab4ac5ecd895f0cdadceb8d95c70"} Jul 23 18:07:41.551043 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:41.550863 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldl42" event={"ID":"f25e2eab-5ed9-4e0b-93c6-9b07325fd6d1","Type":"ContainerStarted","Data":"f0cd237528a842e90d13ad6443c13d5368084ebbd90529ca068a4b4c7e4f46dd"} Jul 23 18:07:41.572312 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:41.571067 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ldl42" podStartSLOduration=68.40511809 podStartE2EDuration="1m9.571047633s" totalImagesPullingTime="1.165929543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:41.570722492 +0000 UTC m=+71.267843568" watchObservedRunningTime="2026-07-23 18:07:41.571047633 +0000 UTC m=+71.268168711" Jul 23 18:07:44.147661 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.147626 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59988fc767-f6sjl"] Jul 23 18:07:44.171393 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.171349 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-5hkdh"] Jul 23 18:07:44.175786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.175760 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5hkdh" Jul 23 18:07:44.178554 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.178530 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 18:07:44.178718 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.178593 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2cmd9\"" Jul 23 18:07:44.178718 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.178530 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 18:07:44.190990 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.190954 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5hkdh"] Jul 23 18:07:44.254853 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.254820 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68ddf7ccf4-6xt8n"] Jul 23 18:07:44.257702 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.257670 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.266540 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.266506 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:44.278807 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.278776 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68ddf7ccf4-6xt8n"] Jul 23 18:07:44.293136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.293101 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx4fr\" (UniqueName: \"kubernetes.io/projected/f17db44b-991d-4c45-8db3-cea8b82259f1-kube-api-access-fx4fr\") pod \"downloads-5bc67dcd46-5hkdh\" (UID: \"f17db44b-991d-4c45-8db3-cea8b82259f1\") " pod="openshift-console/downloads-5bc67dcd46-5hkdh" Jul 23 18:07:44.342590 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.342546 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:44.394332 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394294 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk54p\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-kube-api-access-bk54p\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.394536 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394349 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-ca-trust-extracted\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.394536 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394481 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4fr\" (UniqueName: \"kubernetes.io/projected/f17db44b-991d-4c45-8db3-cea8b82259f1-kube-api-access-fx4fr\") pod \"downloads-5bc67dcd46-5hkdh\" (UID: \"f17db44b-991d-4c45-8db3-cea8b82259f1\") " pod="openshift-console/downloads-5bc67dcd46-5hkdh" Jul 23 18:07:44.394536 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394520 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-bound-sa-token\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.394700 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394570 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-registry-tls\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.394700 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394642 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-image-registry-private-configuration\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.394805 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394726 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-registry-certificates\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.394805 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-trusted-ca\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.394805 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.394792 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-installation-pull-secrets\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.408876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.408799 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx4fr\" (UniqueName: \"kubernetes.io/projected/f17db44b-991d-4c45-8db3-cea8b82259f1-kube-api-access-fx4fr\") pod \"downloads-5bc67dcd46-5hkdh\" (UID: \"f17db44b-991d-4c45-8db3-cea8b82259f1\") " pod="openshift-console/downloads-5bc67dcd46-5hkdh" Jul 23 18:07:44.442053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.442016 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:44.485927 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.485886 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5hkdh" Jul 23 18:07:44.495342 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495302 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-ca-trust-extracted\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495512 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495353 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-bound-sa-token\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495512 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495402 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-registry-tls\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495512 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495439 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-image-registry-private-configuration\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495512 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495475 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-registry-certificates\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495769 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-ca-trust-extracted\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495767 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-trusted-ca\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495842 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-installation-pull-secrets\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.495942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.495904 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk54p\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-kube-api-access-bk54p\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.496750 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.496725 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-registry-certificates\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.496853 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.496814 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-trusted-ca\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.498515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.498491 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-image-registry-private-configuration\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.498745 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.498728 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-registry-tls\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.498911 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.498861 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-installation-pull-secrets\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.509131 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.509100 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk54p\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-kube-api-access-bk54p\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.509256 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.509100 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3-bound-sa-token\") pod \"image-registry-68ddf7ccf4-6xt8n\" (UID: \"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3\") " pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:44.547467 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.547431 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:44.568002 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:44.567968 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:45.350946 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:45.350891 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:45.723949 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:45.723850 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:45.810122 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:45.810099 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:46.668978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:46.668898 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68ddf7ccf4-6xt8n"] Jul 23 18:07:46.674615 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:46.674469 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5hkdh"] Jul 23 18:07:46.678016 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:07:46.677986 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf17db44b_991d_4c45_8db3_cea8b82259f1.slice/crio-7d6ef9a1b7663642af1726900fc36409c97c4f9170f0c61fa22c8dff386a8f5c WatchSource:0}: Error finding container 7d6ef9a1b7663642af1726900fc36409c97c4f9170f0c61fa22c8dff386a8f5c: Status 404 returned error can't find the container with id 7d6ef9a1b7663642af1726900fc36409c97c4f9170f0c61fa22c8dff386a8f5c Jul 23 18:07:46.782288 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:46.782259 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:47.046322 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.046229 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:47.459006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.458970 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:47.463517 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.463488 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f4c7d8fcf-cn2rc"] Jul 23 18:07:47.466804 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.466776 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.469918 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.469612 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 18:07:47.469918 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.469655 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 18:07:47.469918 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.469736 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 18:07:47.470848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.470830 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 18:07:47.470952 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.470900 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f6wlv\"" Jul 23 18:07:47.470952 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.470901 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 18:07:47.480825 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.480683 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4c7d8fcf-cn2rc"] Jul 23 18:07:47.520940 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.520897 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8mnd\" (UniqueName: \"kubernetes.io/projected/cde1979c-0713-442e-b971-eae853148768-kube-api-access-r8mnd\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.521136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.520966 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-oauth-config\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.521136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.520995 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-service-ca\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.521136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.521021 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-console-config\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.521136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.521043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-serving-cert\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.521136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.521099 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-oauth-serving-cert\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.561537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.561496 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:47.582202 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.582163 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9l82s" event={"ID":"1ffe51b0-c8c4-470c-8391-13e33dfca268","Type":"ContainerStarted","Data":"f17c7fa39e901dbdd126317bc7d071ebdc86f60e3d9388d096713e3af4cb494c"} Jul 23 18:07:47.582427 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.582233 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9l82s" event={"ID":"1ffe51b0-c8c4-470c-8391-13e33dfca268","Type":"ContainerStarted","Data":"cf8af383f1afda0c3a7878e4a0652f03b46ba9c2c8943c0c5267f7b9bb365569"} Jul 23 18:07:47.582427 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.582321 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:47.583772 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.583710 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5hkdh" event={"ID":"f17db44b-991d-4c45-8db3-cea8b82259f1","Type":"ContainerStarted","Data":"7d6ef9a1b7663642af1726900fc36409c97c4f9170f0c61fa22c8dff386a8f5c"} Jul 23 18:07:47.586152 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.586121 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" event={"ID":"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3","Type":"ContainerStarted","Data":"7ae14c03031bf971e8c3d16da65277e520da3e73211aa1a4abaab82eaeaea863"} Jul 23 18:07:47.586259 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.586163 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" event={"ID":"d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3","Type":"ContainerStarted","Data":"6a2e6a1df104f5ab9a238fff796446a936c7a2b83525116fcd3e155e4a1213c7"} Jul 23 18:07:47.586330 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.586290 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:07:47.588526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.588501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbwl6" event={"ID":"720a8d7f-6625-4173-acfa-4c6df41f3519","Type":"ContainerStarted","Data":"ff10f3c8d85b81df7158a41ec9b2d260ca6eb283a55b11eda88bcb8e622f77da"} Jul 23 18:07:47.590183 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.590162 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mlbqg" event={"ID":"53440927-f60b-4282-a510-d08f54e11f45","Type":"ContainerStarted","Data":"67efb10fd6decd2256ff080bf052b8ccd5939997e8c4b046a141777bc43021c4"} Jul 23 18:07:47.590307 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.590292 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:07:47.591962 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.591938 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tlbgv" event={"ID":"1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4","Type":"ContainerStarted","Data":"b4763c2e990323aa25ee71d70a2357e6bb820040ea792e25db1d7d4570f89484"} Jul 23 18:07:47.606595 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.606385 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9l82s" podStartSLOduration=34.531829177 podStartE2EDuration="41.606352064s" totalImagesPullingTime="7.074522887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:47.605188349 +0000 UTC m=+77.302309447" watchObservedRunningTime="2026-07-23 18:07:47.606352064 +0000 UTC m=+77.303473142" Jul 23 18:07:47.622295 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.622260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mnd\" (UniqueName: \"kubernetes.io/projected/cde1979c-0713-442e-b971-eae853148768-kube-api-access-r8mnd\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.622479 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.622311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-oauth-config\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.622808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.622561 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-service-ca\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.622808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.622617 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-console-config\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.622808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.622646 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-serving-cert\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.622808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.622717 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-oauth-serving-cert\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.623287 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.623267 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-service-ca\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.623402 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.623325 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-console-config\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.623465 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.623394 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-oauth-serving-cert\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.625460 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.625418 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-oauth-config\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.626277 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.626254 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-serving-cert\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.629409 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.629223 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" podStartSLOduration=3.629207221 podStartE2EDuration="3.629207221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:47.627311116 +0000 UTC m=+77.324432188" watchObservedRunningTime="2026-07-23 18:07:47.629207221 +0000 UTC m=+77.326328297" Jul 23 18:07:47.636269 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.636240 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8mnd\" (UniqueName: \"kubernetes.io/projected/cde1979c-0713-442e-b971-eae853148768-kube-api-access-r8mnd\") pod \"console-7f4c7d8fcf-cn2rc\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.647347 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.647041 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mlbqg" podStartSLOduration=68.542246401 podStartE2EDuration="1m15.647020341s" totalImagesPullingTime="7.10477394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:47.646266021 +0000 UTC m=+77.343387098" watchObservedRunningTime="2026-07-23 18:07:47.647020341 +0000 UTC m=+77.344141418" Jul 23 18:07:47.726429 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.726323 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:47.791052 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.791011 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:47.962098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.961794 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tlbgv" podStartSLOduration=68.837481794 podStartE2EDuration="1m15.961769623s" totalImagesPullingTime="7.124287829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:47.672855387 +0000 UTC m=+77.369976494" watchObservedRunningTime="2026-07-23 18:07:47.961769623 +0000 UTC m=+77.658890696" Jul 23 18:07:47.965585 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.965249 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4c7d8fcf-cn2rc"] Jul 23 18:07:47.987235 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:47.987168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:48.079028 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:48.078995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:48.164730 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:48.164645 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:48.299564 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:48.299477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:48.376756 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:48.376708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:48.598330 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:48.598288 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4c7d8fcf-cn2rc" event={"ID":"cde1979c-0713-442e-b971-eae853148768","Type":"ContainerStarted","Data":"0bffab3687b470e8c0b714de1cc1c15dd26d35fb4f7e1419af1ac69f09c47660"} Jul 23 18:07:49.130570 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:49.130480 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:49.446435 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:49.446275 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:49.546847 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:49.546599 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:49.614076 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:49.614041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:50.859221 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:50.859163 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:50.919774 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:50.919737 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:51.250515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:51.250404 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:51.354321 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:51.354287 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:52.621183 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:52.621043 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sbwl6" event={"ID":"720a8d7f-6625-4173-acfa-4c6df41f3519","Type":"ContainerStarted","Data":"0f96a41df44ca6814ec76c892e7a02a00fdcbe1f66f5eaf19bc0102084012752"} Jul 23 18:07:52.622769 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:52.622739 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4c7d8fcf-cn2rc" event={"ID":"cde1979c-0713-442e-b971-eae853148768","Type":"ContainerStarted","Data":"54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff"} Jul 23 18:07:52.658726 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:52.658585 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sbwl6" podStartSLOduration=17.322150697 podStartE2EDuration="35.658563114s" totalImagesPullingTime="18.336412417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:07:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:52.657689622 +0000 UTC m=+82.354810699" watchObservedRunningTime="2026-07-23 18:07:52.658563114 +0000 UTC m=+82.355684194" Jul 23 18:07:52.692000 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:52.691936 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f4c7d8fcf-cn2rc" podStartSLOduration=1.256493738 podStartE2EDuration="5.691921021s" totalImagesPullingTime="4.435427283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:07:52.691325557 +0000 UTC m=+82.388446631" watchObservedRunningTime="2026-07-23 18:07:52.691921021 +0000 UTC m=+82.389042095" Jul 23 18:07:53.026821 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:53.026719 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:53.102155 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:53.102117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:53.199418 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:53.199383 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:53.297002 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:53.296894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:53.358463 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:53.358431 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:54.154894 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:54.154861 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:07:55.196537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:55.196497 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:55.663027 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:55.662988 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:55.768824 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:55.768788 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:57.217013 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.216971 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:57.430927 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.430887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:57.496569 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.496426 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9bsf9"] Jul 23 18:07:57.500589 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.499131 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.503765 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.503417 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 18:07:57.505253 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.503478 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 18:07:57.505253 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.503613 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 18:07:57.505253 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.503676 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 18:07:57.505253 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.503736 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjcbz\"" Jul 23 18:07:57.505253 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.504249 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 18:07:57.505253 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.504506 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 18:07:57.533977 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.533902 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:57.602555 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.602208 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9l82s" Jul 23 18:07:57.612377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612254 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612304 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-wtmp\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612350 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-textfile\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612402 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd3c7e8f-4252-401d-b14c-adebb8744827-metrics-client-ca\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612431 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-root\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612453 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-sys\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612482 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxrh7\" (UniqueName: \"kubernetes.io/projected/bd3c7e8f-4252-401d-b14c-adebb8744827-kube-api-access-nxrh7\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612533 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-accelerators-collector-config\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.612628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.612559 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.631108 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.631071 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:57.713073 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-accelerators-collector-config\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713284 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713102 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713284 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713150 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713284 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713183 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-wtmp\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713284 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-textfile\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713284 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713259 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd3c7e8f-4252-401d-b14c-adebb8744827-metrics-client-ca\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713563 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-root\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713563 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-sys\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713563 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713355 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxrh7\" (UniqueName: \"kubernetes.io/projected/bd3c7e8f-4252-401d-b14c-adebb8744827-kube-api-access-nxrh7\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.713563 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713549 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-wtmp\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.718196 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713829 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-accelerators-collector-config\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.718196 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713938 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-sys\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.718196 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.713980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd3c7e8f-4252-401d-b14c-adebb8744827-root\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.718196 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:57.714143 2555 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 18:07:57.718196 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.714180 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-textfile\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.718196 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:57.714211 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls podName:bd3c7e8f-4252-401d-b14c-adebb8744827 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:58.214190389 +0000 UTC m=+87.911311446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls") pod "node-exporter-9bsf9" (UID: "bd3c7e8f-4252-401d-b14c-adebb8744827") : secret "node-exporter-tls" not found Jul 23 18:07:57.718196 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.716338 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.718905 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.718086 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd3c7e8f-4252-401d-b14c-adebb8744827-metrics-client-ca\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.728163 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.728129 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxrh7\" (UniqueName: \"kubernetes.io/projected/bd3c7e8f-4252-401d-b14c-adebb8744827-kube-api-access-nxrh7\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:57.791177 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.791084 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:57.791177 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.791163 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:57.797056 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.797025 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:57.804226 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.804183 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:57.907011 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:57.906940 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:58.011293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.011249 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:58.121697 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.121658 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:58.218289 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.218251 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:58.218779 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:58.218448 2555 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 18:07:58.218779 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:58.218526 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls podName:bd3c7e8f-4252-401d-b14c-adebb8744827 nodeName:}" failed. No retries permitted until 2026-07-23 18:07:59.218505755 +0000 UTC m=+88.915626827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls") pod "node-exporter-9bsf9" (UID: "bd3c7e8f-4252-401d-b14c-adebb8744827") : secret "node-exporter-tls" not found Jul 23 18:07:58.327036 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.327000 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:58.428293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.428202 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:58.522866 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.522822 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:58.634397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.634350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:58.638395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.638349 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 18:07:58.642668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.642638 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.645540 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.645505 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 23 18:07:58.645696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.645506 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 23 18:07:58.645696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.645618 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 23 18:07:58.645925 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.645908 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 23 18:07:58.646005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.645932 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 23 18:07:58.646195 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.646179 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 23 18:07:58.646319 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.646284 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 23 18:07:58.646411 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.646336 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 23 18:07:58.646561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.646542 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 23 18:07:58.646640 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.646623 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zc4xc\"" Jul 23 18:07:58.655264 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.655239 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:07:58.665627 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.665590 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 18:07:58.825412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825307 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825354 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825635 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825506 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-web-config\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825635 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825550 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825635 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825575 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8htrl\" (UniqueName: \"kubernetes.io/projected/30b617af-fec5-4698-a93d-e8d21419f2aa-kube-api-access-8htrl\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825758 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825758 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825713 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/30b617af-fec5-4698-a93d-e8d21419f2aa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825758 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825743 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/30b617af-fec5-4698-a93d-e8d21419f2aa-config-out\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825874 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825803 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30b617af-fec5-4698-a93d-e8d21419f2aa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825874 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825866 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.825972 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.825890 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30b617af-fec5-4698-a93d-e8d21419f2aa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.826061 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.826020 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-config-volume\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.826111 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.826059 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/30b617af-fec5-4698-a93d-e8d21419f2aa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.926866 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.926830 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.927063 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.926875 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.927213 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.927178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-web-config\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.927353 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.927230 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.927353 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.927259 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8htrl\" (UniqueName: \"kubernetes.io/projected/30b617af-fec5-4698-a93d-e8d21419f2aa-kube-api-access-8htrl\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.927353 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.927302 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.927555 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:58.927413 2555 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 23 18:07:58.927555 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:07:58.927479 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-main-tls podName:30b617af-fec5-4698-a93d-e8d21419f2aa nodeName:}" failed. No retries permitted until 2026-07-23 18:07:59.427464645 +0000 UTC m=+89.124585697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "30b617af-fec5-4698-a93d-e8d21419f2aa") : secret "alertmanager-main-tls" not found Jul 23 18:07:58.927555 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.927505 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/30b617af-fec5-4698-a93d-e8d21419f2aa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.927555 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.927527 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/30b617af-fec5-4698-a93d-e8d21419f2aa-config-out\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.928260 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.928217 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30b617af-fec5-4698-a93d-e8d21419f2aa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.930395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.929483 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.930395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.929591 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30b617af-fec5-4698-a93d-e8d21419f2aa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.930395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.929772 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-config-volume\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.930395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.929811 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/30b617af-fec5-4698-a93d-e8d21419f2aa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.930395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.930277 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.930717 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.930316 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.932786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.931006 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-web-config\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.932786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.932047 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30b617af-fec5-4698-a93d-e8d21419f2aa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.935038 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.934652 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-config-volume\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.935821 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.935782 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/30b617af-fec5-4698-a93d-e8d21419f2aa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.936112 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.936088 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.936437 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.936403 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30b617af-fec5-4698-a93d-e8d21419f2aa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.937721 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.937695 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.938186 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.938139 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/30b617af-fec5-4698-a93d-e8d21419f2aa-config-out\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.939593 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.939569 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/30b617af-fec5-4698-a93d-e8d21419f2aa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:58.941643 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:58.941617 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8htrl\" (UniqueName: \"kubernetes.io/projected/30b617af-fec5-4698-a93d-e8d21419f2aa-kube-api-access-8htrl\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:59.086580 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.086487 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:07:59.233445 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.233408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:59.236325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.236296 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd3c7e8f-4252-401d-b14c-adebb8744827-node-exporter-tls\") pod \"node-exporter-9bsf9\" (UID: \"bd3c7e8f-4252-401d-b14c-adebb8744827\") " pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:59.313238 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.313198 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9bsf9" Jul 23 18:07:59.435057 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.435018 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:59.437948 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.437919 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/30b617af-fec5-4698-a93d-e8d21419f2aa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"30b617af-fec5-4698-a93d-e8d21419f2aa\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:59.556221 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.556184 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 18:07:59.559296 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.559109 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-768d674c8-jlnhj"] Jul 23 18:07:59.562219 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.562189 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.566171 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.566148 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 18:07:59.566329 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.566314 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 18:07:59.566786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.566764 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 18:07:59.567171 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.567149 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5tjidht6pbddu\"" Jul 23 18:07:59.567278 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.567216 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ddjsv\"" Jul 23 18:07:59.567347 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.567290 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 18:07:59.569835 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.569812 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 18:07:59.592064 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.592026 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-768d674c8-jlnhj"] Jul 23 18:07:59.738200 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738090 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.738200 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738152 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-tls\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.738466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738213 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.738466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738240 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.738466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738337 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.738466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738389 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xqpp\" (UniqueName: \"kubernetes.io/projected/ce395276-f347-41ec-b9e8-f7a90a8da900-kube-api-access-9xqpp\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.738466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738455 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce395276-f347-41ec-b9e8-f7a90a8da900-metrics-client-ca\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.738730 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.738476 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-grpc-tls\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.839769 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.839730 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-tls\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.839959 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.839779 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.839959 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.839804 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.840083 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.840033 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.840083 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.840066 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xqpp\" (UniqueName: \"kubernetes.io/projected/ce395276-f347-41ec-b9e8-f7a90a8da900-kube-api-access-9xqpp\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.840183 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.840151 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce395276-f347-41ec-b9e8-f7a90a8da900-metrics-client-ca\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.840237 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.840187 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-grpc-tls\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.840295 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.840253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.841201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.841146 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce395276-f347-41ec-b9e8-f7a90a8da900-metrics-client-ca\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.843107 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.843079 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.843227 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.843093 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-tls\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.843227 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.843160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.843353 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.843329 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.843543 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.843509 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-grpc-tls\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.843543 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.843531 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ce395276-f347-41ec-b9e8-f7a90a8da900-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.854447 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.854408 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xqpp\" (UniqueName: \"kubernetes.io/projected/ce395276-f347-41ec-b9e8-f7a90a8da900-kube-api-access-9xqpp\") pod \"thanos-querier-768d674c8-jlnhj\" (UID: \"ce395276-f347-41ec-b9e8-f7a90a8da900\") " pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.873362 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.873324 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:07:59.974435 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:07:59.974392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:00.065776 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.065692 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:00.308630 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.308567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:00.405411 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.405340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:00.564532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.564502 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:00.665278 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.665183 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:00.708700 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.708663 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4c7d8fcf-cn2rc"] Jul 23 18:08:00.796528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.796493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:00.893429 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:00.893395 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:01.269755 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:01.269719 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:01.346332 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:01.346296 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:01.781242 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:01.781205 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:01.849307 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:01.849265 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:01.953516 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:01.953477 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx"] Jul 23 18:08:02.113491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.113452 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx"] Jul 23 18:08:02.113693 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.113575 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.118739 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.118712 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 18:08:02.118739 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.118735 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5cmw9\"" Jul 23 18:08:02.118933 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.118753 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 23 18:08:02.118933 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.118764 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e0tjricpjv4a2\"" Jul 23 18:08:02.118933 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.118712 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 23 18:08:02.118933 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.118735 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 23 18:08:02.241883 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.241840 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:02.261697 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.261663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/869f2371-88de-4bf7-abe1-c5a1a27c5efb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.261893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.261719 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mrzh\" (UniqueName: \"kubernetes.io/projected/869f2371-88de-4bf7-abe1-c5a1a27c5efb-kube-api-access-4mrzh\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.261893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.261742 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-secret-metrics-server-client-certs\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.261893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.261799 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-secret-metrics-server-tls\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.262030 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.261904 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/869f2371-88de-4bf7-abe1-c5a1a27c5efb-audit-log\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.262030 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.261957 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/869f2371-88de-4bf7-abe1-c5a1a27c5efb-metrics-server-audit-profiles\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.262030 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.262002 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-client-ca-bundle\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.266152 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.266123 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr"] Jul 23 18:08:02.288830 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.288795 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr"] Jul 23 18:08:02.288981 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.288915 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:02.291757 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.291718 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 23 18:08:02.292070 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.292054 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wdrl8\"" Jul 23 18:08:02.319597 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.319559 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:02.362903 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.362863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-secret-metrics-server-tls\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.363388 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.363019 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/869f2371-88de-4bf7-abe1-c5a1a27c5efb-audit-log\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.363388 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.363052 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/869f2371-88de-4bf7-abe1-c5a1a27c5efb-metrics-server-audit-profiles\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.363388 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.363085 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-client-ca-bundle\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.363388 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.363135 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/869f2371-88de-4bf7-abe1-c5a1a27c5efb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.363388 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.363181 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mrzh\" (UniqueName: \"kubernetes.io/projected/869f2371-88de-4bf7-abe1-c5a1a27c5efb-kube-api-access-4mrzh\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.363388 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.363207 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-secret-metrics-server-client-certs\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.363743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.363515 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/869f2371-88de-4bf7-abe1-c5a1a27c5efb-audit-log\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.364088 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.364060 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/869f2371-88de-4bf7-abe1-c5a1a27c5efb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.364187 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.364086 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/869f2371-88de-4bf7-abe1-c5a1a27c5efb-metrics-server-audit-profiles\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.366127 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.366092 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-client-ca-bundle\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.366286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.366263 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-secret-metrics-server-client-certs\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.366360 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.366320 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/869f2371-88de-4bf7-abe1-c5a1a27c5efb-secret-metrics-server-tls\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.373823 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.373796 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mrzh\" (UniqueName: \"kubernetes.io/projected/869f2371-88de-4bf7-abe1-c5a1a27c5efb-kube-api-access-4mrzh\") pod \"metrics-server-6fbbd8bfc9-n6mjx\" (UID: \"869f2371-88de-4bf7-abe1-c5a1a27c5efb\") " pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.426000 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.425962 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:02.463942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.463909 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tr8gr\" (UID: \"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:02.464096 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.463970 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77fdp\" (UniqueName: \"kubernetes.io/projected/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-kube-api-access-77fdp\") pod \"monitoring-plugin-7b7b7bc8bf-tr8gr\" (UID: \"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:02.565126 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.565086 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fdp\" (UniqueName: \"kubernetes.io/projected/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-kube-api-access-77fdp\") pod \"monitoring-plugin-7b7b7bc8bf-tr8gr\" (UID: \"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:02.565297 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.565186 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tr8gr\" (UID: \"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:02.565297 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:08:02.565283 2555 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 23 18:08:02.565403 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:08:02.565342 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-monitoring-plugin-cert podName:c1ee50f5-e93f-4e63-8aa2-9a61ced0204b nodeName:}" failed. No retries permitted until 2026-07-23 18:08:03.065324419 +0000 UTC m=+92.762445486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-tr8gr" (UID: "c1ee50f5-e93f-4e63-8aa2-9a61ced0204b") : secret "monitoring-plugin-cert" not found Jul 23 18:08:02.575313 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.575284 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77fdp\" (UniqueName: \"kubernetes.io/projected/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-kube-api-access-77fdp\") pod \"monitoring-plugin-7b7b7bc8bf-tr8gr\" (UID: \"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:02.718349 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:02.718261 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:03.069469 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:03.069377 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tr8gr\" (UID: \"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:03.072305 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:03.072277 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c1ee50f5-e93f-4e63-8aa2-9a61ced0204b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tr8gr\" (UID: \"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:03.200303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:03.200259 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:03.896867 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:03.896824 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:04.771497 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:08:04.771462 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3c7e8f_4252_401d_b14c_adebb8744827.slice/crio-dfbb42643047356a7f6aa740bf2d251353cbf278dc3e7abe74caebbfd55bb876 WatchSource:0}: Error finding container dfbb42643047356a7f6aa740bf2d251353cbf278dc3e7abe74caebbfd55bb876: Status 404 returned error can't find the container with id dfbb42643047356a7f6aa740bf2d251353cbf278dc3e7abe74caebbfd55bb876 Jul 23 18:08:04.934235 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:04.934199 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-768d674c8-jlnhj"] Jul 23 18:08:04.963307 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:04.963280 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 18:08:05.101396 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:08:05.101323 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce395276_f347_41ec_b9e8_f7a90a8da900.slice/crio-1449a429faa67372a3c1a445785ee6a5255e6119662c9a4cd9a92c9fdc619422 WatchSource:0}: Error finding container 1449a429faa67372a3c1a445785ee6a5255e6119662c9a4cd9a92c9fdc619422: Status 404 returned error can't find the container with id 1449a429faa67372a3c1a445785ee6a5255e6119662c9a4cd9a92c9fdc619422 Jul 23 18:08:05.101647 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:08:05.101621 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b617af_fec5_4698_a93d_e8d21419f2aa.slice/crio-22f7ae0354942b3c4634fd57e43ac7511d7cdf757c92535f60b42281d90f33ed WatchSource:0}: Error finding container 22f7ae0354942b3c4634fd57e43ac7511d7cdf757c92535f60b42281d90f33ed: Status 404 returned error can't find the container with id 22f7ae0354942b3c4634fd57e43ac7511d7cdf757c92535f60b42281d90f33ed Jul 23 18:08:05.168903 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.168684 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr"] Jul 23 18:08:05.171575 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.171549 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx"] Jul 23 18:08:05.172307 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:08:05.172283 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ee50f5_e93f_4e63_8aa2_9a61ced0204b.slice/crio-c7c80e3648c0617521da4324f66af5ab696c10b0acd32a4068d0f3933b83e222 WatchSource:0}: Error finding container c7c80e3648c0617521da4324f66af5ab696c10b0acd32a4068d0f3933b83e222: Status 404 returned error can't find the container with id c7c80e3648c0617521da4324f66af5ab696c10b0acd32a4068d0f3933b83e222 Jul 23 18:08:05.173816 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:08:05.173794 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869f2371_88de_4bf7_abe1_c5a1a27c5efb.slice/crio-7d3eed3b72a327b12b62082bba663de43fbb840cd950ee8bef5abb0d01a39fa8 WatchSource:0}: Error finding container 7d3eed3b72a327b12b62082bba663de43fbb840cd950ee8bef5abb0d01a39fa8: Status 404 returned error can't find the container with id 7d3eed3b72a327b12b62082bba663de43fbb840cd950ee8bef5abb0d01a39fa8 Jul 23 18:08:05.665250 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.665181 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:05.681556 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.681518 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bsf9" event={"ID":"bd3c7e8f-4252-401d-b14c-adebb8744827","Type":"ContainerStarted","Data":"dfbb42643047356a7f6aa740bf2d251353cbf278dc3e7abe74caebbfd55bb876"} Jul 23 18:08:05.683551 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.683518 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" event={"ID":"869f2371-88de-4bf7-abe1-c5a1a27c5efb","Type":"ContainerStarted","Data":"7d3eed3b72a327b12b62082bba663de43fbb840cd950ee8bef5abb0d01a39fa8"} Jul 23 18:08:05.688122 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.686009 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" event={"ID":"ce395276-f347-41ec-b9e8-f7a90a8da900","Type":"ContainerStarted","Data":"1449a429faa67372a3c1a445785ee6a5255e6119662c9a4cd9a92c9fdc619422"} Jul 23 18:08:05.694540 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.694469 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" event={"ID":"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b","Type":"ContainerStarted","Data":"c7c80e3648c0617521da4324f66af5ab696c10b0acd32a4068d0f3933b83e222"} Jul 23 18:08:05.696389 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.696336 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerStarted","Data":"22f7ae0354942b3c4634fd57e43ac7511d7cdf757c92535f60b42281d90f33ed"} Jul 23 18:08:05.699414 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.699389 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5hkdh" event={"ID":"f17db44b-991d-4c45-8db3-cea8b82259f1","Type":"ContainerStarted","Data":"4ef16571052425c67033cd3746bbfc83593e4146b6fe19c0caa7171bf2a1d2e7"} Jul 23 18:08:05.700736 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.700661 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-5hkdh" Jul 23 18:08:05.713922 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.713857 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-5hkdh" Jul 23 18:08:05.736764 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.736528 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-5hkdh" podStartSLOduration=3.234730399 podStartE2EDuration="21.736509558s" totalImagesPullingTime="18.501779159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:05.736091758 +0000 UTC m=+95.433212847" watchObservedRunningTime="2026-07-23 18:08:05.736509558 +0000 UTC m=+95.433630676" Jul 23 18:08:05.748510 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:05.748448 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:06.710396 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:06.710117 2555 generic.go:356] "Generic (PLEG): container finished" podID="bd3c7e8f-4252-401d-b14c-adebb8744827" containerID="151c47e62bfd72d9340b736febc08638fe3ddd4fb4f4737eb6dd54fd41672f4e" exitCode=0 Jul 23 18:08:06.710396 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:06.710278 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bsf9" event={"ID":"bd3c7e8f-4252-401d-b14c-adebb8744827","Type":"ContainerDied","Data":"151c47e62bfd72d9340b736febc08638fe3ddd4fb4f4737eb6dd54fd41672f4e"} Jul 23 18:08:06.776137 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:06.775991 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:06.851320 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:06.851253 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:08.604394 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:08.604058 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68ddf7ccf4-6xt8n" Jul 23 18:08:08.791329 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:08.791288 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:08.880076 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:08.879954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:08.983137 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:08.983079 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:09.170037 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.169960 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" podUID="d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" containerName="registry" containerID="cri-o://a029b60bbca1b472a017bbc18b21d766a4d00ececc21729c3e8ab4c985c23143" gracePeriod=30 Jul 23 18:08:09.725766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.725535 2555 generic.go:356] "Generic (PLEG): container finished" podID="d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" containerID="a029b60bbca1b472a017bbc18b21d766a4d00ececc21729c3e8ab4c985c23143" exitCode=0 Jul 23 18:08:09.725766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.725596 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" event={"ID":"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2","Type":"ContainerDied","Data":"a029b60bbca1b472a017bbc18b21d766a4d00ececc21729c3e8ab4c985c23143"} Jul 23 18:08:09.725766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.725700 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" event={"ID":"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2","Type":"ContainerDied","Data":"b9b14d8f8808ddc525083702352f130ba91829ce07f4fe459cc28bef2d4f1dca"} Jul 23 18:08:09.725766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.725723 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9b14d8f8808ddc525083702352f130ba91829ce07f4fe459cc28bef2d4f1dca" Jul 23 18:08:09.727812 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.727788 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bsf9" event={"ID":"bd3c7e8f-4252-401d-b14c-adebb8744827","Type":"ContainerStarted","Data":"9fa698780c18f0db3b1dad907feee26cbacc792ab5e91f59679fdd28bdd3fc72"} Jul 23 18:08:09.804070 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.803797 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:08:09.826743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826718 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-image-registry-private-configuration\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.826852 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826757 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-installation-pull-secrets\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.826852 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826783 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-bound-sa-token\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.826852 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826818 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-trusted-ca\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.827014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826866 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h48nl\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-kube-api-access-h48nl\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.827014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826897 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.827014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826935 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-ca-trust-extracted\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.827014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.826986 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-certificates\") pod \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\" (UID: \"d70b858b-5f3b-4b5e-b55e-b03f5e7691c2\") " Jul 23 18:08:09.828573 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.827668 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:08:09.828573 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.827706 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:08:09.829660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.829630 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:08:09.833244 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.833073 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:08:09.835415 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.835388 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:08:09.836756 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.836713 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:08:09.837538 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.837497 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-kube-api-access-h48nl" (OuterVolumeSpecName: "kube-api-access-h48nl") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "kube-api-access-h48nl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:08:09.841613 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.841578 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" (UID: "d70b858b-5f3b-4b5e-b55e-b03f5e7691c2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928085 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h48nl\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-kube-api-access-h48nl\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928119 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-tls\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928134 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-ca-trust-extracted\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928148 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-registry-certificates\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928164 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-image-registry-private-configuration\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928178 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-installation-pull-secrets\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928193 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-bound-sa-token\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:09.928233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:09.928206 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2-trusted-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:10.228613 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.228573 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:10.300656 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.300573 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:10.734529 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.734491 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bsf9" event={"ID":"bd3c7e8f-4252-401d-b14c-adebb8744827","Type":"ContainerStarted","Data":"6edf6dcd520566227e3666c6a5451a6f2ec6e7ce4a09572bb2caa2b472e7ef19"} Jul 23 18:08:10.736322 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.736284 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" event={"ID":"869f2371-88de-4bf7-abe1-c5a1a27c5efb","Type":"ContainerStarted","Data":"f5f271400415b767d36e009ecf3f0fcfccadab6ff260bddbeb5c8bf2d386147a"} Jul 23 18:08:10.738707 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.738677 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" event={"ID":"ce395276-f347-41ec-b9e8-f7a90a8da900","Type":"ContainerStarted","Data":"afe900d01e2cece9e2c58d7286018307dae2b7a8c57d9b160fcf3396807e0fe2"} Jul 23 18:08:10.738837 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.738751 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" event={"ID":"ce395276-f347-41ec-b9e8-f7a90a8da900","Type":"ContainerStarted","Data":"9ce34e2a790c116e36d4183e02fd9864d886a99d0c3727033b8309ef147701ff"} Jul 23 18:08:10.738837 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.738795 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" event={"ID":"ce395276-f347-41ec-b9e8-f7a90a8da900","Type":"ContainerStarted","Data":"51b2e280ce7a69aa80a1a01df72da591ac5b9ab765e76abb9ab15c184425e366"} Jul 23 18:08:10.740407 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.740356 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" event={"ID":"c1ee50f5-e93f-4e63-8aa2-9a61ced0204b","Type":"ContainerStarted","Data":"f2941b1b8c3d59b30c2b5db39f63bbf948a50b6cdadba9f4dd9af3c9e777bb14"} Jul 23 18:08:10.740513 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.740430 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:10.742053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.742013 2555 generic.go:356] "Generic (PLEG): container finished" podID="30b617af-fec5-4698-a93d-e8d21419f2aa" containerID="5c5951c2938615b2b84986396acc15d565ce6a7eb5bc05bded14e53bd14b2ec4" exitCode=0 Jul 23 18:08:10.742148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.742075 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59988fc767-f6sjl" Jul 23 18:08:10.742148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.742074 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerDied","Data":"5c5951c2938615b2b84986396acc15d565ce6a7eb5bc05bded14e53bd14b2ec4"} Jul 23 18:08:10.746955 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.746934 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" Jul 23 18:08:10.783140 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.783075 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9bsf9" podStartSLOduration=12.644496923 podStartE2EDuration="13.783054459s" totalImagesPullingTime="1.008729572s" totalInitContainerRuntime="129.827964ms" isStatefulPod=true podCreationTimestamp="2026-07-23 18:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:10.782126237 +0000 UTC m=+100.479247334" watchObservedRunningTime="2026-07-23 18:08:10.783054459 +0000 UTC m=+100.480175568" Jul 23 18:08:10.930141 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.930067 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tr8gr" podStartSLOduration=4.529450409 podStartE2EDuration="8.930042313s" totalImagesPullingTime="4.400591904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:10.906101129 +0000 UTC m=+100.603222202" watchObservedRunningTime="2026-07-23 18:08:10.930042313 +0000 UTC m=+100.627163391" Jul 23 18:08:10.931845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.931794 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59988fc767-f6sjl"] Jul 23 18:08:10.938082 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.938055 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59988fc767-f6sjl"] Jul 23 18:08:10.967589 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:10.967523 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" podStartSLOduration=5.568936386 podStartE2EDuration="9.967500791s" totalImagesPullingTime="4.398564405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:10.963165493 +0000 UTC m=+100.660286567" watchObservedRunningTime="2026-07-23 18:08:10.967500791 +0000 UTC m=+100.664621865" Jul 23 18:08:12.248851 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:12.248738 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" path="/var/lib/kubelet/pods/d70b858b-5f3b-4b5e-b55e-b03f5e7691c2/volumes" Jul 23 18:08:12.756271 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:12.756225 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" event={"ID":"ce395276-f347-41ec-b9e8-f7a90a8da900","Type":"ContainerStarted","Data":"96def92d895e93703592e586ebd382b61979833363dc228982a17bea18894abf"} Jul 23 18:08:12.756466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:12.756310 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" event={"ID":"ce395276-f347-41ec-b9e8-f7a90a8da900","Type":"ContainerStarted","Data":"aebbb3a3ee9c5a3049f31da4dcfd823fb63c1f201f0a31bc7b982da710495cdd"} Jul 23 18:08:12.756466 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:12.756354 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" event={"ID":"ce395276-f347-41ec-b9e8-f7a90a8da900","Type":"ContainerStarted","Data":"cab10201c44e2703bdcf76c361ff7360336b50527d415d0205b3138edbc9feed"} Jul 23 18:08:12.789268 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:12.789200 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" podStartSLOduration=7.803965589 podStartE2EDuration="13.789179292s" totalImagesPullingTime="5.985213703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:12.788070958 +0000 UTC m=+102.485192077" watchObservedRunningTime="2026-07-23 18:08:12.789179292 +0000 UTC m=+102.486300366" Jul 23 18:08:13.766133 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.766094 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerStarted","Data":"3ab7e6ae14e422735cf9f34b124f5c724e319e024e8b32d5d7acf03a1d09f0c5"} Jul 23 18:08:13.766610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.766178 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerStarted","Data":"080062ce24c4bb60e0e3ee258721bdecbe76b0f4a928c0ea780efe083ab3de59"} Jul 23 18:08:13.766610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.766221 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerStarted","Data":"f5a366a6424b783e16c6d84482b171fc9e21b31aac8c342d4ff7eee5e86932db"} Jul 23 18:08:13.766610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.766251 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerStarted","Data":"e5fc6d31a0546a3f9e143405bd0a62745804ce443d52e645711795ef5161485b"} Jul 23 18:08:13.766610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.766279 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerStarted","Data":"aae15daa3b63773715bc5f327f407e54d2a6a6a6651d82aaf2a55e9cea39b1ac"} Jul 23 18:08:13.766610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.766318 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"30b617af-fec5-4698-a93d-e8d21419f2aa","Type":"ContainerStarted","Data":"6ae03dc69f806e4c324079b52130c21ebeea4ada7a1ffbafcfe75150cff470c4"} Jul 23 18:08:13.766610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.766597 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:08:13.802252 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:13.802141 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.061296231 podStartE2EDuration="15.802120343s" totalImagesPullingTime="6.714245095s" totalInitContainerRuntime="26.579017ms" isStatefulPod=false podCreationTimestamp="2026-07-23 18:07:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:13.80188217 +0000 UTC m=+103.499003246" watchObservedRunningTime="2026-07-23 18:08:13.802120343 +0000 UTC m=+103.499241418" Jul 23 18:08:15.830711 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:15.830673 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:15.910905 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:15.910873 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:16.800274 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:16.800225 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:16.897688 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:16.897654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:18.600921 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:18.600891 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mlbqg" Jul 23 18:08:19.776110 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:19.776081 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-768d674c8-jlnhj" Jul 23 18:08:19.946679 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:19.946646 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:20.041998 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:20.041907 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:20.145491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:20.145452 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:22.426826 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:22.426772 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:22.427283 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:22.426841 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:24.487263 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:24.487222 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:24.608045 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:24.608004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:24.711981 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:24.711943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:25.733508 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:25.733465 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f4c7d8fcf-cn2rc" podUID="cde1979c-0713-442e-b971-eae853148768" containerName="console" containerID="cri-o://54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff" gracePeriod=15 Jul 23 18:08:26.007873 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.007807 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:08:26.070418 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070361 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-serving-cert\") pod \"cde1979c-0713-442e-b971-eae853148768\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " Jul 23 18:08:26.070418 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070420 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8mnd\" (UniqueName: \"kubernetes.io/projected/cde1979c-0713-442e-b971-eae853148768-kube-api-access-r8mnd\") pod \"cde1979c-0713-442e-b971-eae853148768\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " Jul 23 18:08:26.070660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070449 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-oauth-serving-cert\") pod \"cde1979c-0713-442e-b971-eae853148768\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " Jul 23 18:08:26.070660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070468 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-service-ca\") pod \"cde1979c-0713-442e-b971-eae853148768\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " Jul 23 18:08:26.070660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070488 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-oauth-config\") pod \"cde1979c-0713-442e-b971-eae853148768\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " Jul 23 18:08:26.070660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070524 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-console-config\") pod \"cde1979c-0713-442e-b971-eae853148768\" (UID: \"cde1979c-0713-442e-b971-eae853148768\") " Jul 23 18:08:26.070955 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070911 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-service-ca" (OuterVolumeSpecName: "service-ca") pod "cde1979c-0713-442e-b971-eae853148768" (UID: "cde1979c-0713-442e-b971-eae853148768"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:08:26.070955 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070921 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cde1979c-0713-442e-b971-eae853148768" (UID: "cde1979c-0713-442e-b971-eae853148768"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:08:26.071140 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.070954 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-console-config" (OuterVolumeSpecName: "console-config") pod "cde1979c-0713-442e-b971-eae853148768" (UID: "cde1979c-0713-442e-b971-eae853148768"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:08:26.072822 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.072793 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cde1979c-0713-442e-b971-eae853148768" (UID: "cde1979c-0713-442e-b971-eae853148768"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:08:26.072909 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.072825 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde1979c-0713-442e-b971-eae853148768-kube-api-access-r8mnd" (OuterVolumeSpecName: "kube-api-access-r8mnd") pod "cde1979c-0713-442e-b971-eae853148768" (UID: "cde1979c-0713-442e-b971-eae853148768"). InnerVolumeSpecName "kube-api-access-r8mnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:08:26.072909 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.072859 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cde1979c-0713-442e-b971-eae853148768" (UID: "cde1979c-0713-442e-b971-eae853148768"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:08:26.171444 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.171407 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:26.171444 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.171437 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-oauth-config\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:26.171444 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.171451 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-console-config\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:26.171669 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.171460 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde1979c-0713-442e-b971-eae853148768-console-serving-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:26.171669 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.171469 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8mnd\" (UniqueName: \"kubernetes.io/projected/cde1979c-0713-442e-b971-eae853148768-kube-api-access-r8mnd\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:26.171669 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.171477 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde1979c-0713-442e-b971-eae853148768-oauth-serving-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:08:26.815529 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.815491 2555 generic.go:356] "Generic (PLEG): container finished" podID="cde1979c-0713-442e-b971-eae853148768" containerID="54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff" exitCode=0 Jul 23 18:08:26.815987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.815544 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4c7d8fcf-cn2rc" Jul 23 18:08:26.815987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.815561 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4c7d8fcf-cn2rc" event={"ID":"cde1979c-0713-442e-b971-eae853148768","Type":"ContainerDied","Data":"54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff"} Jul 23 18:08:26.815987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.815612 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4c7d8fcf-cn2rc" event={"ID":"cde1979c-0713-442e-b971-eae853148768","Type":"ContainerDied","Data":"0bffab3687b470e8c0b714de1cc1c15dd26d35fb4f7e1419af1ac69f09c47660"} Jul 23 18:08:26.815987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.815640 2555 scope.go:117] "RemoveContainer" containerID="54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff" Jul 23 18:08:26.831738 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.831720 2555 scope.go:117] "RemoveContainer" containerID="54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff" Jul 23 18:08:26.832052 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:08:26.832021 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff\": container with ID starting with 54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff not found: ID does not exist" containerID="54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff" Jul 23 18:08:26.832106 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.832062 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff"} err="failed to get container status \"54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff\": rpc error: code = NotFound desc = could not find container \"54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff\": container with ID starting with 54d328b22732c1329e6d3ffaa572e1f966829d25212f4f2c99f11a145a8edeff not found: ID does not exist" Jul 23 18:08:26.848523 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.845005 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4c7d8fcf-cn2rc"] Jul 23 18:08:26.851141 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:26.851116 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f4c7d8fcf-cn2rc"] Jul 23 18:08:28.245971 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:28.245935 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde1979c-0713-442e-b971-eae853148768" path="/var/lib/kubelet/pods/cde1979c-0713-442e-b971-eae853148768/volumes" Jul 23 18:08:31.787875 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:31.787836 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:31.906003 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:31.905975 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:42.432004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:42.431973 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:42.435876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:42.435851 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fbbd8bfc9-n6mjx" Jul 23 18:08:56.290932 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:56.290894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:08:56.492377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:08:56.492335 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:13.778132 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:13.778092 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:13.873358 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:13.873322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:20.444130 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.444092 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6685644756-xwzns"] Jul 23 18:09:20.444532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.444412 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde1979c-0713-442e-b971-eae853148768" containerName="console" Jul 23 18:09:20.444532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.444428 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde1979c-0713-442e-b971-eae853148768" containerName="console" Jul 23 18:09:20.444532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.444445 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" containerName="registry" Jul 23 18:09:20.444532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.444457 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" containerName="registry" Jul 23 18:09:20.444532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.444515 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d70b858b-5f3b-4b5e-b55e-b03f5e7691c2" containerName="registry" Jul 23 18:09:20.444532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.444524 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde1979c-0713-442e-b971-eae853148768" containerName="console" Jul 23 18:09:20.447395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.447355 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.451699 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.451675 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 18:09:20.452827 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.452780 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f6wlv\"" Jul 23 18:09:20.452938 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.452886 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 18:09:20.453113 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.453093 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 18:09:20.454171 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.453968 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 18:09:20.454566 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.454547 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 18:09:20.456768 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.456750 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 18:09:20.462199 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.462177 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6685644756-xwzns"] Jul 23 18:09:20.515558 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.515525 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-serving-cert\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.515711 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.515560 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-oauth-config\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.515711 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.515594 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-oauth-serving-cert\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.515711 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.515649 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hzt\" (UniqueName: \"kubernetes.io/projected/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-kube-api-access-q4hzt\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.515711 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.515681 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-config\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.515711 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.515705 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-trusted-ca-bundle\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.515899 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.515768 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-service-ca\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.523833 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.523808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:20.616557 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.616522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-serving-cert\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.616557 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.616560 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-oauth-config\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.616786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.616594 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-oauth-serving-cert\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.616786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.616626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hzt\" (UniqueName: \"kubernetes.io/projected/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-kube-api-access-q4hzt\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.616786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.616657 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-config\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.616786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.616678 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-trusted-ca-bundle\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.616786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.616729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-service-ca\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.617478 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.617452 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-service-ca\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.617606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.617482 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-config\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.617606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.617490 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-oauth-serving-cert\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.617752 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.617733 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-trusted-ca-bundle\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.619031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.619013 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-serving-cert\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.619125 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.619073 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-oauth-config\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.626047 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.626026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:20.628152 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.628131 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hzt\" (UniqueName: \"kubernetes.io/projected/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-kube-api-access-q4hzt\") pod \"console-6685644756-xwzns\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.689106 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.689085 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:20.757385 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.757298 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:20.889292 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:20.889268 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6685644756-xwzns"] Jul 23 18:09:20.891970 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:09:20.891948 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1eef56_891b_4e63_8ddb_9dfbd9f470a6.slice/crio-9455074e421ed578abedaf7c7133259893b1fa33afee189e8d56f1c0ce0ef28e WatchSource:0}: Error finding container 9455074e421ed578abedaf7c7133259893b1fa33afee189e8d56f1c0ce0ef28e: Status 404 returned error can't find the container with id 9455074e421ed578abedaf7c7133259893b1fa33afee189e8d56f1c0ce0ef28e Jul 23 18:09:21.021232 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:21.021094 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6685644756-xwzns" event={"ID":"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6","Type":"ContainerStarted","Data":"c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725"} Jul 23 18:09:21.021232 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:21.021184 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6685644756-xwzns" event={"ID":"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6","Type":"ContainerStarted","Data":"9455074e421ed578abedaf7c7133259893b1fa33afee189e8d56f1c0ce0ef28e"} Jul 23 18:09:21.046447 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:21.046399 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6685644756-xwzns" podStartSLOduration=1.046382397 podStartE2EDuration="1.046382397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:09:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:09:21.045734437 +0000 UTC m=+170.742855558" watchObservedRunningTime="2026-07-23 18:09:21.046382397 +0000 UTC m=+170.743503470" Jul 23 18:09:21.227496 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:21.227453 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:21.328612 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:21.328533 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:21.834680 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:21.834647 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:21.906644 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:21.906612 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:25.734849 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:25.734760 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:25.839238 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:25.839207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:30.757627 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:30.757576 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:30.758147 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:30.757660 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:30.762249 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:30.762222 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:31.060932 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:31.060850 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:09:31.256218 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:31.256167 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:31.356910 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:31.356877 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:31.430258 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:31.430227 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:31.780400 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:31.780289 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:31.880294 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:31.880263 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:31.981428 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:31.981394 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:33.248275 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:33.248228 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:33.351049 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:33.351016 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:58.334651 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:58.334612 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:09:58.409030 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:09:58.408986 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:26.238088 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:26.238049 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:26.338574 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:26.338538 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:26.466616 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:26.466582 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:53.192264 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:53.192157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:54.223910 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:54.223869 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:57.076121 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:57.076087 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:57.170570 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:57.170534 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:58.737560 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:58.737526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:58.861311 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:58.861272 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:59.217243 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:59.217203 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:10:59.323257 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:10:59.323228 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:00.506154 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.506124 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-45jlg"] Jul 23 18:11:00.508150 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.508133 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.510904 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.510877 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 18:11:00.511026 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.510921 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sbvwn\"" Jul 23 18:11:00.511732 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.511714 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 18:11:00.518478 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.518458 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-45jlg"] Jul 23 18:11:00.549666 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.549620 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dxzv\" (UniqueName: \"kubernetes.io/projected/f60c17f0-543c-4b2d-909b-4012120e6b1a-kube-api-access-2dxzv\") pod \"cert-manager-webhook-6486ffd997-45jlg\" (UID: \"f60c17f0-543c-4b2d-909b-4012120e6b1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.549666 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.549664 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f60c17f0-543c-4b2d-909b-4012120e6b1a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-45jlg\" (UID: \"f60c17f0-543c-4b2d-909b-4012120e6b1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.650633 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.650576 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dxzv\" (UniqueName: \"kubernetes.io/projected/f60c17f0-543c-4b2d-909b-4012120e6b1a-kube-api-access-2dxzv\") pod \"cert-manager-webhook-6486ffd997-45jlg\" (UID: \"f60c17f0-543c-4b2d-909b-4012120e6b1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.650633 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.650634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f60c17f0-543c-4b2d-909b-4012120e6b1a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-45jlg\" (UID: \"f60c17f0-543c-4b2d-909b-4012120e6b1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.661458 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.661425 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dxzv\" (UniqueName: \"kubernetes.io/projected/f60c17f0-543c-4b2d-909b-4012120e6b1a-kube-api-access-2dxzv\") pod \"cert-manager-webhook-6486ffd997-45jlg\" (UID: \"f60c17f0-543c-4b2d-909b-4012120e6b1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.664494 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.664471 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f60c17f0-543c-4b2d-909b-4012120e6b1a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-45jlg\" (UID: \"f60c17f0-543c-4b2d-909b-4012120e6b1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.816828 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.816737 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:00.956010 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:00.955981 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-45jlg"] Jul 23 18:11:00.959052 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:11:00.959028 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf60c17f0_543c_4b2d_909b_4012120e6b1a.slice/crio-b28b8c0235f785175fcd2ae997f2e890241e65cf8816073b43d4e5a1c3711420 WatchSource:0}: Error finding container b28b8c0235f785175fcd2ae997f2e890241e65cf8816073b43d4e5a1c3711420: Status 404 returned error can't find the container with id b28b8c0235f785175fcd2ae997f2e890241e65cf8816073b43d4e5a1c3711420 Jul 23 18:11:01.394929 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:01.394892 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" event={"ID":"f60c17f0-543c-4b2d-909b-4012120e6b1a","Type":"ContainerStarted","Data":"b28b8c0235f785175fcd2ae997f2e890241e65cf8816073b43d4e5a1c3711420"} Jul 23 18:11:01.594087 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:01.594026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:01.839946 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:01.839919 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:03.802963 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:03.802930 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:03.887168 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:03.887140 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:09.209250 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.209176 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:09.427519 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.427476 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" event={"ID":"f60c17f0-543c-4b2d-909b-4012120e6b1a","Type":"ContainerStarted","Data":"ba3c719c37bed574e3d503c20111cf4969e1c7cc470f8717d77f7ded6ce7e206"} Jul 23 18:11:09.427655 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.427556 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:09.451212 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.451165 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" podStartSLOduration=1.4526673460000001 podStartE2EDuration="9.451153334s" totalImagesPullingTime="7.998485988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:11:09.449756217 +0000 UTC m=+279.146877303" watchObservedRunningTime="2026-07-23 18:11:09.451153334 +0000 UTC m=+279.148274408" Jul 23 18:11:09.862003 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.861969 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4l5jn"] Jul 23 18:11:09.864058 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.864043 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:09.866515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.866495 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-z4fqp\"" Jul 23 18:11:09.886322 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.886295 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4l5jn"] Jul 23 18:11:09.914408 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.914381 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z58rj\" (UniqueName: \"kubernetes.io/projected/72c2b3c1-7770-47c6-948c-889cc158da93-kube-api-access-z58rj\") pod \"cert-manager-55b4d954d5-4l5jn\" (UID: \"72c2b3c1-7770-47c6-948c-889cc158da93\") " pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:09.914567 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.914439 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72c2b3c1-7770-47c6-948c-889cc158da93-bound-sa-token\") pod \"cert-manager-55b4d954d5-4l5jn\" (UID: \"72c2b3c1-7770-47c6-948c-889cc158da93\") " pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:09.942901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:09.942858 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:10.015302 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.015265 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z58rj\" (UniqueName: \"kubernetes.io/projected/72c2b3c1-7770-47c6-948c-889cc158da93-kube-api-access-z58rj\") pod \"cert-manager-55b4d954d5-4l5jn\" (UID: \"72c2b3c1-7770-47c6-948c-889cc158da93\") " pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:10.015495 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.015345 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72c2b3c1-7770-47c6-948c-889cc158da93-bound-sa-token\") pod \"cert-manager-55b4d954d5-4l5jn\" (UID: \"72c2b3c1-7770-47c6-948c-889cc158da93\") " pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:10.029903 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.029874 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72c2b3c1-7770-47c6-948c-889cc158da93-bound-sa-token\") pod \"cert-manager-55b4d954d5-4l5jn\" (UID: \"72c2b3c1-7770-47c6-948c-889cc158da93\") " pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:10.030024 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.029947 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z58rj\" (UniqueName: \"kubernetes.io/projected/72c2b3c1-7770-47c6-948c-889cc158da93-kube-api-access-z58rj\") pod \"cert-manager-55b4d954d5-4l5jn\" (UID: \"72c2b3c1-7770-47c6-948c-889cc158da93\") " pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:10.037584 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.037559 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:10.172197 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.172110 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4l5jn" Jul 23 18:11:10.300974 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.300943 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4l5jn"] Jul 23 18:11:10.303834 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:11:10.303805 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72c2b3c1_7770_47c6_948c_889cc158da93.slice/crio-61ac3dbe7de396fec2a8eef5465ace2f5ab4831175ac0fd1ce36dd16247c7552 WatchSource:0}: Error finding container 61ac3dbe7de396fec2a8eef5465ace2f5ab4831175ac0fd1ce36dd16247c7552: Status 404 returned error can't find the container with id 61ac3dbe7de396fec2a8eef5465ace2f5ab4831175ac0fd1ce36dd16247c7552 Jul 23 18:11:10.433148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.433060 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4l5jn" event={"ID":"72c2b3c1-7770-47c6-948c-889cc158da93","Type":"ContainerStarted","Data":"30eb8170f37e8659124c78669c888e17d0a92af56bfca53c74023b3efcb9fbe0"} Jul 23 18:11:10.433148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.433107 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4l5jn" event={"ID":"72c2b3c1-7770-47c6-948c-889cc158da93","Type":"ContainerStarted","Data":"61ac3dbe7de396fec2a8eef5465ace2f5ab4831175ac0fd1ce36dd16247c7552"} Jul 23 18:11:10.452978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:10.452930 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4l5jn" podStartSLOduration=1.452891269 podStartE2EDuration="1.452891269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:11:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:11:10.451439678 +0000 UTC m=+280.148560754" watchObservedRunningTime="2026-07-23 18:11:10.452891269 +0000 UTC m=+280.150012344" Jul 23 18:11:11.284262 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:11.284178 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:11.381551 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:11.381513 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:12.749146 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.749111 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg"] Jul 23 18:11:12.751132 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.751117 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:12.756163 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.756136 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-48fg9\"" Jul 23 18:11:12.756499 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.756485 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 18:11:12.756720 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.756704 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 18:11:12.786153 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.786130 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg"] Jul 23 18:11:12.839085 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.839050 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f39d568-9384-48da-bf7e-039f634d3ee8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ndnmg\" (UID: \"4f39d568-9384-48da-bf7e-039f634d3ee8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:12.839243 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.839113 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4txd8\" (UniqueName: \"kubernetes.io/projected/4f39d568-9384-48da-bf7e-039f634d3ee8-kube-api-access-4txd8\") pod \"openshift-lws-operator-fd8ccff4c-ndnmg\" (UID: \"4f39d568-9384-48da-bf7e-039f634d3ee8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:12.940255 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.940218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f39d568-9384-48da-bf7e-039f634d3ee8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ndnmg\" (UID: \"4f39d568-9384-48da-bf7e-039f634d3ee8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:12.940430 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.940307 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4txd8\" (UniqueName: \"kubernetes.io/projected/4f39d568-9384-48da-bf7e-039f634d3ee8-kube-api-access-4txd8\") pod \"openshift-lws-operator-fd8ccff4c-ndnmg\" (UID: \"4f39d568-9384-48da-bf7e-039f634d3ee8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:12.940620 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.940598 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f39d568-9384-48da-bf7e-039f634d3ee8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ndnmg\" (UID: \"4f39d568-9384-48da-bf7e-039f634d3ee8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:12.963584 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:12.963552 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4txd8\" (UniqueName: \"kubernetes.io/projected/4f39d568-9384-48da-bf7e-039f634d3ee8-kube-api-access-4txd8\") pod \"openshift-lws-operator-fd8ccff4c-ndnmg\" (UID: \"4f39d568-9384-48da-bf7e-039f634d3ee8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:13.060603 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:13.060529 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" Jul 23 18:11:13.199741 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:13.199653 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg"] Jul 23 18:11:13.202351 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:11:13.202322 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f39d568_9384_48da_bf7e_039f634d3ee8.slice/crio-3f1e6986274e00924e8e50628bbde75d5d1c57888dc064a24e1f6084048eeb77 WatchSource:0}: Error finding container 3f1e6986274e00924e8e50628bbde75d5d1c57888dc064a24e1f6084048eeb77: Status 404 returned error can't find the container with id 3f1e6986274e00924e8e50628bbde75d5d1c57888dc064a24e1f6084048eeb77 Jul 23 18:11:13.445413 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:13.445354 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" event={"ID":"4f39d568-9384-48da-bf7e-039f634d3ee8","Type":"ContainerStarted","Data":"3f1e6986274e00924e8e50628bbde75d5d1c57888dc064a24e1f6084048eeb77"} Jul 23 18:11:15.435651 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:15.435619 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-45jlg" Jul 23 18:11:16.460726 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:16.460684 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" event={"ID":"4f39d568-9384-48da-bf7e-039f634d3ee8","Type":"ContainerStarted","Data":"d8037855ce423e7a8dcbf96da9b1a5ae100e124d64e16ad2d5a99b67716b8294"} Jul 23 18:11:16.485715 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:16.485664 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ndnmg" podStartSLOduration=1.788820656 podStartE2EDuration="4.485649424s" totalImagesPullingTime="2.696828768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:11:16.483496111 +0000 UTC m=+286.180617218" watchObservedRunningTime="2026-07-23 18:11:16.485649424 +0000 UTC m=+286.182770497" Jul 23 18:11:17.319146 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:17.319111 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:17.411090 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:17.411053 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:18.989876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:18.989841 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:31.921544 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:31.921514 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:11:31.922090 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:31.921599 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:11:31.929208 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:31.929181 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 18:11:35.234229 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:35.234171 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:35.339835 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:35.339801 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:35.434452 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:35.434416 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:35.867660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:35.867627 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:36.382452 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.382407 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:36.477128 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.477081 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:36.517452 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.517418 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg"] Jul 23 18:11:36.520600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.520582 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.523389 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.523343 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 18:11:36.523528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.523388 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 18:11:36.523528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.523407 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 18:11:36.523528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.523391 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kmncx\"" Jul 23 18:11:36.523528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.523483 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 18:11:36.544423 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.544393 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg"] Jul 23 18:11:36.627625 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.627588 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h77pp\" (UniqueName: \"kubernetes.io/projected/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-kube-api-access-h77pp\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.627625 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.627630 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-tmp\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.627842 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.627649 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.627842 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.627736 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-manifests\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.627842 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.627774 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.728511 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.728414 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-manifests\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.728511 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.728464 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.728725 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.728532 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h77pp\" (UniqueName: \"kubernetes.io/projected/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-kube-api-access-h77pp\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.728725 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.728550 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-tmp\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.728725 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.728569 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.728915 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.728889 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-manifests\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.730967 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.730946 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.731177 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.731162 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-tmp\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.731524 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.731503 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.748526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.748494 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:36.749672 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.749648 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h77pp\" (UniqueName: \"kubernetes.io/projected/61f5815f-6367-43e9-82c3-cb5e5a14ed5d-kube-api-access-h77pp\") pod \"opendatahub-operator-controller-manager-56b5877c86-zczlg\" (UID: \"61f5815f-6367-43e9-82c3-cb5e5a14ed5d\") " pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.830894 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.830858 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:36.843099 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.843066 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:36.945186 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.945157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:36.973198 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.973175 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg"] Jul 23 18:11:36.976011 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:11:36.975983 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f5815f_6367_43e9_82c3_cb5e5a14ed5d.slice/crio-6d38d37a5d53a1b6d178a9273bda77af4bab080f1b6b1a748f09ded4bc783833 WatchSource:0}: Error finding container 6d38d37a5d53a1b6d178a9273bda77af4bab080f1b6b1a748f09ded4bc783833: Status 404 returned error can't find the container with id 6d38d37a5d53a1b6d178a9273bda77af4bab080f1b6b1a748f09ded4bc783833 Jul 23 18:11:36.977890 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:36.977871 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:11:37.022978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:37.022941 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:37.100349 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:37.100315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:37.198011 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:37.197978 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:37.300071 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:37.299988 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:37.555094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:37.555009 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" event={"ID":"61f5815f-6367-43e9-82c3-cb5e5a14ed5d","Type":"ContainerStarted","Data":"6d38d37a5d53a1b6d178a9273bda77af4bab080f1b6b1a748f09ded4bc783833"} Jul 23 18:11:40.577277 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:40.577239 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" event={"ID":"61f5815f-6367-43e9-82c3-cb5e5a14ed5d","Type":"ContainerStarted","Data":"b52f0a9b76c2ae55e88082761a90367c0d339bfe0e1be09fcb4e853dbb690864"} Jul 23 18:11:41.583004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:41.582913 2555 generic.go:356] "Generic (PLEG): container finished" podID="61f5815f-6367-43e9-82c3-cb5e5a14ed5d" containerID="b52f0a9b76c2ae55e88082761a90367c0d339bfe0e1be09fcb4e853dbb690864" exitCode=0 Jul 23 18:11:41.583004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:41.582982 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" event={"ID":"61f5815f-6367-43e9-82c3-cb5e5a14ed5d","Type":"ContainerDied","Data":"b52f0a9b76c2ae55e88082761a90367c0d339bfe0e1be09fcb4e853dbb690864"} Jul 23 18:11:42.589385 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:42.589335 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" event={"ID":"61f5815f-6367-43e9-82c3-cb5e5a14ed5d","Type":"ContainerStarted","Data":"d3b7df32cfb17176628b40a784d2e98e9e6e07c9032d6d8ddbd7a0198c1358f5"} Jul 23 18:11:42.589839 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:42.589443 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:42.627294 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:42.627233 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" podStartSLOduration=2.111162078 podStartE2EDuration="6.627214743s" totalImagesPullingTime="3.440510245s" totalInitContainerRuntime="1.07554242s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:11:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:11:42.624283224 +0000 UTC m=+312.321404332" watchObservedRunningTime="2026-07-23 18:11:42.627214743 +0000 UTC m=+312.324335827" Jul 23 18:11:53.597036 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:53.597006 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56b5877c86-zczlg" Jul 23 18:11:55.856831 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:55.856797 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:11:56.035966 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:11:56.035934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:01.703139 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.703102 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:01.795230 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.795195 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m"] Jul 23 18:12:01.798507 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.798488 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:01.809032 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.809012 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fmsdv\"" Jul 23 18:12:01.809171 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.809151 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 18:12:01.809528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.809511 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 18:12:01.809592 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.809567 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 18:12:01.809918 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.809890 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 18:12:01.821290 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.821258 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m"] Jul 23 18:12:01.891187 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.891143 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c6n28"] Jul 23 18:12:01.894964 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.894942 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:01.898537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.898519 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 18:12:01.898537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.898530 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tmkgb\"" Jul 23 18:12:01.908121 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.908092 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c6n28"] Jul 23 18:12:01.924141 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.924109 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:01.942850 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.942817 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb373434-fc26-4b41-bb3c-0bb56dd65624-tls-certs\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:01.943042 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.942876 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l79px\" (UniqueName: \"kubernetes.io/projected/cb373434-fc26-4b41-bb3c-0bb56dd65624-kube-api-access-l79px\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:01.943042 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:01.942906 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb373434-fc26-4b41-bb3c-0bb56dd65624-tmp\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.019152 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.019034 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:02.043448 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.043415 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l79px\" (UniqueName: \"kubernetes.io/projected/cb373434-fc26-4b41-bb3c-0bb56dd65624-kube-api-access-l79px\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.043646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.043455 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe0d6b30-6243-44b6-abc0-8532682e23e1-cert\") pod \"odh-model-controller-7998c67dc4-c6n28\" (UID: \"fe0d6b30-6243-44b6-abc0-8532682e23e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.043646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.043479 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb373434-fc26-4b41-bb3c-0bb56dd65624-tmp\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.043646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.043568 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrv4k\" (UniqueName: \"kubernetes.io/projected/fe0d6b30-6243-44b6-abc0-8532682e23e1-kube-api-access-qrv4k\") pod \"odh-model-controller-7998c67dc4-c6n28\" (UID: \"fe0d6b30-6243-44b6-abc0-8532682e23e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.043812 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.043644 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb373434-fc26-4b41-bb3c-0bb56dd65624-tls-certs\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.045891 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.045857 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb373434-fc26-4b41-bb3c-0bb56dd65624-tmp\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.046090 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.046072 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb373434-fc26-4b41-bb3c-0bb56dd65624-tls-certs\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.064676 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.064601 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l79px\" (UniqueName: \"kubernetes.io/projected/cb373434-fc26-4b41-bb3c-0bb56dd65624-kube-api-access-l79px\") pod \"kube-auth-proxy-f87ff99bf-8v88m\" (UID: \"cb373434-fc26-4b41-bb3c-0bb56dd65624\") " pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.108549 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.108507 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" Jul 23 18:12:02.119951 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.119912 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:02.144999 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.144955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe0d6b30-6243-44b6-abc0-8532682e23e1-cert\") pod \"odh-model-controller-7998c67dc4-c6n28\" (UID: \"fe0d6b30-6243-44b6-abc0-8532682e23e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.145199 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.145022 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrv4k\" (UniqueName: \"kubernetes.io/projected/fe0d6b30-6243-44b6-abc0-8532682e23e1-kube-api-access-qrv4k\") pod \"odh-model-controller-7998c67dc4-c6n28\" (UID: \"fe0d6b30-6243-44b6-abc0-8532682e23e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.145395 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:12:02.145351 2555 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 23 18:12:02.145478 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:12:02.145452 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe0d6b30-6243-44b6-abc0-8532682e23e1-cert podName:fe0d6b30-6243-44b6-abc0-8532682e23e1 nodeName:}" failed. No retries permitted until 2026-07-23 18:12:02.645430846 +0000 UTC m=+332.342551897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fe0d6b30-6243-44b6-abc0-8532682e23e1-cert") pod "odh-model-controller-7998c67dc4-c6n28" (UID: "fe0d6b30-6243-44b6-abc0-8532682e23e1") : secret "odh-model-controller-webhook-cert" not found Jul 23 18:12:02.156518 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.156306 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrv4k\" (UniqueName: \"kubernetes.io/projected/fe0d6b30-6243-44b6-abc0-8532682e23e1-kube-api-access-qrv4k\") pod \"odh-model-controller-7998c67dc4-c6n28\" (UID: \"fe0d6b30-6243-44b6-abc0-8532682e23e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.269772 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.269748 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m"] Jul 23 18:12:02.272337 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:12:02.272310 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb373434_fc26_4b41_bb3c_0bb56dd65624.slice/crio-4acf1773f09f998f97312426890e45575b3e423229f0b5f45c50e1daf8a6894e WatchSource:0}: Error finding container 4acf1773f09f998f97312426890e45575b3e423229f0b5f45c50e1daf8a6894e: Status 404 returned error can't find the container with id 4acf1773f09f998f97312426890e45575b3e423229f0b5f45c50e1daf8a6894e Jul 23 18:12:02.291940 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.291902 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:02.648486 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.648448 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe0d6b30-6243-44b6-abc0-8532682e23e1-cert\") pod \"odh-model-controller-7998c67dc4-c6n28\" (UID: \"fe0d6b30-6243-44b6-abc0-8532682e23e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.650923 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.650899 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe0d6b30-6243-44b6-abc0-8532682e23e1-cert\") pod \"odh-model-controller-7998c67dc4-c6n28\" (UID: \"fe0d6b30-6243-44b6-abc0-8532682e23e1\") " pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.679360 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.679326 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" event={"ID":"cb373434-fc26-4b41-bb3c-0bb56dd65624","Type":"ContainerStarted","Data":"4acf1773f09f998f97312426890e45575b3e423229f0b5f45c50e1daf8a6894e"} Jul 23 18:12:02.731801 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.731768 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:02.805907 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.805867 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:02.954978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.954948 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c6n28"] Jul 23 18:12:02.956682 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:12:02.956630 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0d6b30_6243_44b6_abc0_8532682e23e1.slice/crio-1dfdc2e3a7ddae38756f01efa0508c8b2ad42525255dcbc408c506ade257078e WatchSource:0}: Error finding container 1dfdc2e3a7ddae38756f01efa0508c8b2ad42525255dcbc408c506ade257078e: Status 404 returned error can't find the container with id 1dfdc2e3a7ddae38756f01efa0508c8b2ad42525255dcbc408c506ade257078e Jul 23 18:12:02.990815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.990782 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d92g9"] Jul 23 18:12:02.995993 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:02.995965 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.006617 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.006581 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qxll2\"" Jul 23 18:12:03.006754 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.006622 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 18:12:03.008954 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.008929 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d92g9"] Jul 23 18:12:03.153913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.153869 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2cd7\" (UniqueName: \"kubernetes.io/projected/091d8ecb-d73b-4a7e-99a1-f53101a0d347-kube-api-access-r2cd7\") pod \"model-serving-api-54d89bd5fc-d92g9\" (UID: \"091d8ecb-d73b-4a7e-99a1-f53101a0d347\") " pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.154105 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.153991 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/091d8ecb-d73b-4a7e-99a1-f53101a0d347-tls-certs\") pod \"model-serving-api-54d89bd5fc-d92g9\" (UID: \"091d8ecb-d73b-4a7e-99a1-f53101a0d347\") " pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.164460 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.164427 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:03.164689 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.164668 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-gf45p"] Jul 23 18:12:03.168485 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.168467 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.173040 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.173012 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 18:12:03.173040 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.173026 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 18:12:03.173270 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.173016 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 18:12:03.173270 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.173043 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 18:12:03.173412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.173320 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s8prq\"" Jul 23 18:12:03.189919 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.189879 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-gf45p"] Jul 23 18:12:03.241808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.241697 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:03.255588 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255534 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/091d8ecb-d73b-4a7e-99a1-f53101a0d347-tls-certs\") pod \"model-serving-api-54d89bd5fc-d92g9\" (UID: \"091d8ecb-d73b-4a7e-99a1-f53101a0d347\") " pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.255773 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255644 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.255773 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-cacerts\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.255773 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255736 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.255936 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255784 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmx4\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-kube-api-access-njmx4\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.255936 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255827 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2cd7\" (UniqueName: \"kubernetes.io/projected/091d8ecb-d73b-4a7e-99a1-f53101a0d347-kube-api-access-r2cd7\") pod \"model-serving-api-54d89bd5fc-d92g9\" (UID: \"091d8ecb-d73b-4a7e-99a1-f53101a0d347\") " pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.255936 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255852 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.255936 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255922 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-local-certs\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.256134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.255980 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-token\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.258265 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.258237 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/091d8ecb-d73b-4a7e-99a1-f53101a0d347-tls-certs\") pod \"model-serving-api-54d89bd5fc-d92g9\" (UID: \"091d8ecb-d73b-4a7e-99a1-f53101a0d347\") " pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.266824 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.266786 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2cd7\" (UniqueName: \"kubernetes.io/projected/091d8ecb-d73b-4a7e-99a1-f53101a0d347-kube-api-access-r2cd7\") pod \"model-serving-api-54d89bd5fc-d92g9\" (UID: \"091d8ecb-d73b-4a7e-99a1-f53101a0d347\") " pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.307022 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.306991 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:03.339165 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.339135 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:03.357476 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.357380 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.357476 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.357426 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-local-certs\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.357476 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.357449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-token\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.357742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.357497 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.357742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.357523 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-cacerts\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.357742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.357537 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.357742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.357557 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njmx4\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-kube-api-access-njmx4\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.359068 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.358756 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.361039 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.361011 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-cacerts\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.361160 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.361011 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.361481 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.361461 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.361637 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.361622 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-local-certs\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.370495 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.370424 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-token\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.370704 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.370682 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmx4\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-kube-api-access-njmx4\") pod \"istiod-openshift-gateway-7574459656-gf45p\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.463035 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.462970 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-d92g9"] Jul 23 18:12:03.465877 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:12:03.465845 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091d8ecb_d73b_4a7e_99a1_f53101a0d347.slice/crio-6e21ed8c5d497e51aa651da1acbb64e7ce32e20ba863367215f45788d064bd0d WatchSource:0}: Error finding container 6e21ed8c5d497e51aa651da1acbb64e7ce32e20ba863367215f45788d064bd0d: Status 404 returned error can't find the container with id 6e21ed8c5d497e51aa651da1acbb64e7ce32e20ba863367215f45788d064bd0d Jul 23 18:12:03.478014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.477986 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:03.648271 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.648205 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-gf45p"] Jul 23 18:12:03.686028 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.685985 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" event={"ID":"fe0d6b30-6243-44b6-abc0-8532682e23e1","Type":"ContainerStarted","Data":"1dfdc2e3a7ddae38756f01efa0508c8b2ad42525255dcbc408c506ade257078e"} Jul 23 18:12:03.687644 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.687612 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" event={"ID":"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1","Type":"ContainerStarted","Data":"71fdca8e84d08826b7e074e8eda33ac734b24dd15355989de933fb6102c3aa8b"} Jul 23 18:12:03.689088 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:03.689060 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" event={"ID":"091d8ecb-d73b-4a7e-99a1-f53101a0d347","Type":"ContainerStarted","Data":"6e21ed8c5d497e51aa651da1acbb64e7ce32e20ba863367215f45788d064bd0d"} Jul 23 18:12:04.421214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:04.421174 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:08.458457 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.458408 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:12:08.458752 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.458493 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:12:08.719496 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.719405 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" event={"ID":"cb373434-fc26-4b41-bb3c-0bb56dd65624","Type":"ContainerStarted","Data":"6f17980775fcd2e95137697251f80b88a31ba4435e99072031ede95da77cff41"} Jul 23 18:12:08.721414 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.721356 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" event={"ID":"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1","Type":"ContainerStarted","Data":"a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559"} Jul 23 18:12:08.721538 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.721432 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:08.723136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.723111 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" event={"ID":"091d8ecb-d73b-4a7e-99a1-f53101a0d347","Type":"ContainerStarted","Data":"77fd968fd575efdb97e330a51e577357bbb4925ec34df5d9e9637276a5559001"} Jul 23 18:12:08.723255 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.723219 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:08.724785 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.724763 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" event={"ID":"fe0d6b30-6243-44b6-abc0-8532682e23e1","Type":"ContainerStarted","Data":"a41a3caaf5fc5b2069a8451ba47ea684367a334964cff664fbe7a31a1d21d5ba"} Jul 23 18:12:08.724909 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.724892 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:08.742282 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.742232 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f87ff99bf-8v88m" podStartSLOduration=1.5703835339999999 podStartE2EDuration="7.742215579s" totalImagesPullingTime="6.171832045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:08.741424968 +0000 UTC m=+338.438546069" watchObservedRunningTime="2026-07-23 18:12:08.742215579 +0000 UTC m=+338.439336657" Jul 23 18:12:08.767198 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.767146 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" podStartSLOduration=2.272099112 podStartE2EDuration="7.767129204s" totalImagesPullingTime="5.495030092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:08.765635236 +0000 UTC m=+338.462756310" watchObservedRunningTime="2026-07-23 18:12:08.767129204 +0000 UTC m=+338.464250280" Jul 23 18:12:08.786005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.785959 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" podStartSLOduration=1.7572555589999999 podStartE2EDuration="6.785945518s" totalImagesPullingTime="5.028689959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:08.784484463 +0000 UTC m=+338.481605541" watchObservedRunningTime="2026-07-23 18:12:08.785945518 +0000 UTC m=+338.483066591" Jul 23 18:12:08.810490 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:08.810438 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" podStartSLOduration=1.006099572 podStartE2EDuration="5.81042027s" totalImagesPullingTime="4.804320698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:08.808098381 +0000 UTC m=+338.505219479" watchObservedRunningTime="2026-07-23 18:12:08.81042027 +0000 UTC m=+338.507541346" Jul 23 18:12:09.464400 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:09.464351 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:09.558134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:09.558099 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:09.731161 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:09.731071 2555 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-gf45p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 18:12:09.731161 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:09.731134 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" podUID="faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 18:12:10.589374 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:10.589326 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:10.819201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:10.819167 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:10.915657 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:10.915579 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:12.730396 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:12.730341 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:14.634283 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.634251 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:14.728175 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.728142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:14.792292 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.792257 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr"] Jul 23 18:12:14.799688 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.799655 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:14.804916 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.804865 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 18:12:14.805061 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.805009 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vbgqm\"" Jul 23 18:12:14.821317 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.821289 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds"] Jul 23 18:12:14.838668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.838637 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr"] Jul 23 18:12:14.838808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.838786 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:14.851251 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.851224 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 23 18:12:14.851380 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.851328 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rqltf\"" Jul 23 18:12:14.856709 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.856683 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds"] Jul 23 18:12:14.862742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.862713 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ec0c80-04e0-4bca-a4d8-b7f6093c36df-cert\") pod \"kserve-controller-manager-d6f4b65b8-wvdqr\" (UID: \"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:14.862846 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.862756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs6xj\" (UniqueName: \"kubernetes.io/projected/b1ec0c80-04e0-4bca-a4d8-b7f6093c36df-kube-api-access-vs6xj\") pod \"kserve-controller-manager-d6f4b65b8-wvdqr\" (UID: \"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:14.867132 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.867112 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:14.963701 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.963607 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzhsh\" (UniqueName: \"kubernetes.io/projected/7730a224-8f32-49c2-9217-7f25304b9f8e-kube-api-access-rzhsh\") pod \"llmisvc-controller-manager-777c6bd7d5-4fgds\" (UID: \"7730a224-8f32-49c2-9217-7f25304b9f8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:14.963701 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.963649 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7730a224-8f32-49c2-9217-7f25304b9f8e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4fgds\" (UID: \"7730a224-8f32-49c2-9217-7f25304b9f8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:14.963908 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.963738 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ec0c80-04e0-4bca-a4d8-b7f6093c36df-cert\") pod \"kserve-controller-manager-d6f4b65b8-wvdqr\" (UID: \"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:14.963908 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.963771 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs6xj\" (UniqueName: \"kubernetes.io/projected/b1ec0c80-04e0-4bca-a4d8-b7f6093c36df-kube-api-access-vs6xj\") pod \"kserve-controller-manager-d6f4b65b8-wvdqr\" (UID: \"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:14.966345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.966314 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ec0c80-04e0-4bca-a4d8-b7f6093c36df-cert\") pod \"kserve-controller-manager-d6f4b65b8-wvdqr\" (UID: \"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:14.979257 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:14.979231 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs6xj\" (UniqueName: \"kubernetes.io/projected/b1ec0c80-04e0-4bca-a4d8-b7f6093c36df-kube-api-access-vs6xj\") pod \"kserve-controller-manager-d6f4b65b8-wvdqr\" (UID: \"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:15.022204 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.022154 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:15.064219 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.064186 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzhsh\" (UniqueName: \"kubernetes.io/projected/7730a224-8f32-49c2-9217-7f25304b9f8e-kube-api-access-rzhsh\") pod \"llmisvc-controller-manager-777c6bd7d5-4fgds\" (UID: \"7730a224-8f32-49c2-9217-7f25304b9f8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:15.064219 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.064220 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7730a224-8f32-49c2-9217-7f25304b9f8e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4fgds\" (UID: \"7730a224-8f32-49c2-9217-7f25304b9f8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:15.064483 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:12:15.064411 2555 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 23 18:12:15.064483 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:12:15.064477 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7730a224-8f32-49c2-9217-7f25304b9f8e-cert podName:7730a224-8f32-49c2-9217-7f25304b9f8e nodeName:}" failed. No retries permitted until 2026-07-23 18:12:15.564459578 +0000 UTC m=+345.261580647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7730a224-8f32-49c2-9217-7f25304b9f8e-cert") pod "llmisvc-controller-manager-777c6bd7d5-4fgds" (UID: "7730a224-8f32-49c2-9217-7f25304b9f8e") : secret "llmisvc-webhook-server-cert" not found Jul 23 18:12:15.079214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.079184 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzhsh\" (UniqueName: \"kubernetes.io/projected/7730a224-8f32-49c2-9217-7f25304b9f8e-kube-api-access-rzhsh\") pod \"llmisvc-controller-manager-777c6bd7d5-4fgds\" (UID: \"7730a224-8f32-49c2-9217-7f25304b9f8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:15.113056 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.113021 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:15.193155 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.193117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:15.272623 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.272595 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr"] Jul 23 18:12:15.275381 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:12:15.275339 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ec0c80_04e0_4bca_a4d8_b7f6093c36df.slice/crio-0f22381021ea9afdc5d3f4710f419cb665ba3067bd626a72bf51d70b8fabe20b WatchSource:0}: Error finding container 0f22381021ea9afdc5d3f4710f419cb665ba3067bd626a72bf51d70b8fabe20b: Status 404 returned error can't find the container with id 0f22381021ea9afdc5d3f4710f419cb665ba3067bd626a72bf51d70b8fabe20b Jul 23 18:12:15.569402 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.569280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7730a224-8f32-49c2-9217-7f25304b9f8e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4fgds\" (UID: \"7730a224-8f32-49c2-9217-7f25304b9f8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:15.571676 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.571646 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7730a224-8f32-49c2-9217-7f25304b9f8e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4fgds\" (UID: \"7730a224-8f32-49c2-9217-7f25304b9f8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:15.750590 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.750554 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:15.757960 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.757923 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" event={"ID":"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df","Type":"ContainerStarted","Data":"0f22381021ea9afdc5d3f4710f419cb665ba3067bd626a72bf51d70b8fabe20b"} Jul 23 18:12:15.895040 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:15.895012 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds"] Jul 23 18:12:15.897279 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:12:15.897248 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7730a224_8f32_49c2_9217_7f25304b9f8e.slice/crio-9935f12cf89f2daac763622422c623753c7a45ad02a426e48cad1a9bf0a9b260 WatchSource:0}: Error finding container 9935f12cf89f2daac763622422c623753c7a45ad02a426e48cad1a9bf0a9b260: Status 404 returned error can't find the container with id 9935f12cf89f2daac763622422c623753c7a45ad02a426e48cad1a9bf0a9b260 Jul 23 18:12:16.144502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:16.144471 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:16.252939 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:16.252895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:16.672901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:16.672860 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-gf45p"] Jul 23 18:12:16.673132 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:16.673102 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" podUID="faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" containerName="discovery" containerID="cri-o://a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559" gracePeriod=30 Jul 23 18:12:16.766190 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:16.766151 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" event={"ID":"7730a224-8f32-49c2-9217-7f25304b9f8e","Type":"ContainerStarted","Data":"9935f12cf89f2daac763622422c623753c7a45ad02a426e48cad1a9bf0a9b260"} Jul 23 18:12:17.106314 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.106284 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:17.182906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.182829 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-local-certs\") pod \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " Jul 23 18:12:17.183078 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.182912 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-ca-configmap\") pod \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " Jul 23 18:12:17.183078 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.182950 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-token\") pod \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " Jul 23 18:12:17.183078 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.182989 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njmx4\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-kube-api-access-njmx4\") pod \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " Jul 23 18:12:17.183078 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.183019 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-kubeconfig\") pod \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " Jul 23 18:12:17.183078 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.183043 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-dns-cert\") pod \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " Jul 23 18:12:17.183326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.183108 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-cacerts\") pod \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\" (UID: \"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1\") " Jul 23 18:12:17.183401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.183354 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" (UID: "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:12:17.194548 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.194301 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-cacerts" (OuterVolumeSpecName: "cacerts") pod "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" (UID: "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:12:17.194548 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.194411 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" (UID: "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:12:17.194548 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.194493 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-local-certs" (OuterVolumeSpecName: "local-certs") pod "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" (UID: "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:12:17.196574 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.196397 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" (UID: "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:12:17.196574 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.196546 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" (UID: "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:12:17.197007 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.196964 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-kube-api-access-njmx4" (OuterVolumeSpecName: "kube-api-access-njmx4") pod "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" (UID: "faa31caa-bbac-4f44-a4bd-7a3b77acb4a1"). InnerVolumeSpecName "kube-api-access-njmx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:12:17.284315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.284181 2555 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-cacerts\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:12:17.284315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.284217 2555 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-local-certs\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:12:17.284315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.284234 2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-ca-configmap\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:12:17.284315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.284247 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-token\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:12:17.284315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.284261 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njmx4\" (UniqueName: \"kubernetes.io/projected/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-kube-api-access-njmx4\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:12:17.284315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.284274 2555 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-kubeconfig\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:12:17.284315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.284290 2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1-istio-csr-dns-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:12:17.773513 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.773473 2555 generic.go:356] "Generic (PLEG): container finished" podID="faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" containerID="a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559" exitCode=0 Jul 23 18:12:17.773994 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.773529 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" Jul 23 18:12:17.773994 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.773595 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" event={"ID":"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1","Type":"ContainerDied","Data":"a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559"} Jul 23 18:12:17.773994 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.773647 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-gf45p" event={"ID":"faa31caa-bbac-4f44-a4bd-7a3b77acb4a1","Type":"ContainerDied","Data":"71fdca8e84d08826b7e074e8eda33ac734b24dd15355989de933fb6102c3aa8b"} Jul 23 18:12:17.773994 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.773676 2555 scope.go:117] "RemoveContainer" containerID="a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559" Jul 23 18:12:17.800587 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.800559 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-gf45p"] Jul 23 18:12:17.810697 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.810673 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-gf45p"] Jul 23 18:12:17.866805 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.866775 2555 scope.go:117] "RemoveContainer" containerID="a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559" Jul 23 18:12:17.867199 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:12:17.867164 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559\": container with ID starting with a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559 not found: ID does not exist" containerID="a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559" Jul 23 18:12:17.867294 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:17.867213 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559"} err="failed to get container status \"a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559\": rpc error: code = NotFound desc = could not find container \"a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559\": container with ID starting with a2dbe91c031d4bdc200593b0246899eaacf3cf4315f1b0d539fe36fb0adc8559 not found: ID does not exist" Jul 23 18:12:18.250575 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:18.250535 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" path="/var/lib/kubelet/pods/faa31caa-bbac-4f44-a4bd-7a3b77acb4a1/volumes" Jul 23 18:12:18.780378 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:18.780337 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" event={"ID":"b1ec0c80-04e0-4bca-a4d8-b7f6093c36df","Type":"ContainerStarted","Data":"dd56fabbfb6abf821fcabebc0571a4d7ab2b1fe07744a500127317ccd80dc0df"} Jul 23 18:12:18.780815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:18.780426 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:18.781797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:18.781741 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" event={"ID":"7730a224-8f32-49c2-9217-7f25304b9f8e","Type":"ContainerStarted","Data":"ea9cecb3ea0a6790d4ae4bdb4acdc443c99baa929ae1d03c2d132ce8019feef6"} Jul 23 18:12:18.781862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:18.781833 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:18.803710 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:18.803656 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" podStartSLOduration=2.206711296 podStartE2EDuration="4.803635125s" totalImagesPullingTime="2.596923829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:18.80213321 +0000 UTC m=+348.499254280" watchObservedRunningTime="2026-07-23 18:12:18.803635125 +0000 UTC m=+348.500756189" Jul 23 18:12:18.830142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:18.830092 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" podStartSLOduration=2.195978692 podStartE2EDuration="4.83007498s" totalImagesPullingTime="2.634096288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:18.828872885 +0000 UTC m=+348.525993959" watchObservedRunningTime="2026-07-23 18:12:18.83007498 +0000 UTC m=+348.527196054" Jul 23 18:12:19.731923 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:19.731892 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-c6n28" Jul 23 18:12:19.733614 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:19.733593 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-d92g9" Jul 23 18:12:23.786461 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:23.786382 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:23.885037 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:23.885000 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:49.788178 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:49.788136 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4fgds" Jul 23 18:12:49.791005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:49.790984 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvdqr" Jul 23 18:12:57.529334 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.529300 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:57.618084 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.618045 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:57.640387 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.640331 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tfzzc"] Jul 23 18:12:57.640718 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.640705 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" containerName="discovery" Jul 23 18:12:57.640768 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.640719 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" containerName="discovery" Jul 23 18:12:57.640801 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.640796 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="faa31caa-bbac-4f44-a4bd-7a3b77acb4a1" containerName="discovery" Jul 23 18:12:57.642977 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.642955 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" Jul 23 18:12:57.645675 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.645646 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 18:12:57.646600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.646573 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 18:12:57.646600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.646579 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-v52kl\"" Jul 23 18:12:57.656345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.656320 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tfzzc"] Jul 23 18:12:57.696124 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.696097 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:57.736094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.736051 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md7nl\" (UniqueName: \"kubernetes.io/projected/3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff-kube-api-access-md7nl\") pod \"kuadrant-operator-catalog-tfzzc\" (UID: \"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff\") " pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" Jul 23 18:12:57.837451 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.837332 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md7nl\" (UniqueName: \"kubernetes.io/projected/3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff-kube-api-access-md7nl\") pod \"kuadrant-operator-catalog-tfzzc\" (UID: \"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff\") " pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" Jul 23 18:12:57.847324 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.847290 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md7nl\" (UniqueName: \"kubernetes.io/projected/3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff-kube-api-access-md7nl\") pod \"kuadrant-operator-catalog-tfzzc\" (UID: \"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff\") " pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" Jul 23 18:12:57.953697 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:57.953663 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" Jul 23 18:12:58.000165 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.000130 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tfzzc"] Jul 23 18:12:58.092491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.092466 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tfzzc"] Jul 23 18:12:58.095191 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:12:58.095151 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad08de5_e4a1_4a70_a2e5_ef4a535c30ff.slice/crio-3f402b67011b6a95ce1be120ac7d29560b2a95a4b22e9a59fd304dc970bf9e1d WatchSource:0}: Error finding container 3f402b67011b6a95ce1be120ac7d29560b2a95a4b22e9a59fd304dc970bf9e1d: Status 404 returned error can't find the container with id 3f402b67011b6a95ce1be120ac7d29560b2a95a4b22e9a59fd304dc970bf9e1d Jul 23 18:12:58.209281 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.209242 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fptmb"] Jul 23 18:12:58.212195 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.212177 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:12:58.221636 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.221612 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fptmb"] Jul 23 18:12:58.240780 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.240755 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45z69\" (UniqueName: \"kubernetes.io/projected/a3bd66e9-308b-4d5c-a03d-11b11193c4fa-kube-api-access-45z69\") pod \"kuadrant-operator-catalog-fptmb\" (UID: \"a3bd66e9-308b-4d5c-a03d-11b11193c4fa\") " pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:12:58.342117 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.342025 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45z69\" (UniqueName: \"kubernetes.io/projected/a3bd66e9-308b-4d5c-a03d-11b11193c4fa-kube-api-access-45z69\") pod \"kuadrant-operator-catalog-fptmb\" (UID: \"a3bd66e9-308b-4d5c-a03d-11b11193c4fa\") " pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:12:58.353164 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.353129 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45z69\" (UniqueName: \"kubernetes.io/projected/a3bd66e9-308b-4d5c-a03d-11b11193c4fa-kube-api-access-45z69\") pod \"kuadrant-operator-catalog-fptmb\" (UID: \"a3bd66e9-308b-4d5c-a03d-11b11193c4fa\") " pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:12:58.443121 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.443087 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:58.522067 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.522028 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:12:58.642505 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.642350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:12:58.669896 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.669836 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fptmb"] Jul 23 18:12:58.680470 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:12:58.680439 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3bd66e9_308b_4d5c_a03d_11b11193c4fa.slice/crio-f247ec32fc7719385ea60e121273e28cc88fdfd11aaf7bcdad1dfa8f313faac3 WatchSource:0}: Error finding container f247ec32fc7719385ea60e121273e28cc88fdfd11aaf7bcdad1dfa8f313faac3: Status 404 returned error can't find the container with id f247ec32fc7719385ea60e121273e28cc88fdfd11aaf7bcdad1dfa8f313faac3 Jul 23 18:12:58.966123 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.966042 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" event={"ID":"a3bd66e9-308b-4d5c-a03d-11b11193c4fa","Type":"ContainerStarted","Data":"f247ec32fc7719385ea60e121273e28cc88fdfd11aaf7bcdad1dfa8f313faac3"} Jul 23 18:12:58.967201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:12:58.967173 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" event={"ID":"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff","Type":"ContainerStarted","Data":"3f402b67011b6a95ce1be120ac7d29560b2a95a4b22e9a59fd304dc970bf9e1d"} Jul 23 18:13:00.978953 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:00.978910 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" event={"ID":"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff","Type":"ContainerStarted","Data":"c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8"} Jul 23 18:13:00.978953 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:00.978947 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" podUID="3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff" containerName="registry-server" containerID="cri-o://c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8" gracePeriod=2 Jul 23 18:13:00.980526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:00.980493 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" event={"ID":"a3bd66e9-308b-4d5c-a03d-11b11193c4fa","Type":"ContainerStarted","Data":"ffed74d157fb2346c97e758a196ad7b43093c15a69f5a0d2f451a4075af70b01"} Jul 23 18:13:01.029927 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.029878 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" podStartSLOduration=1.984050863 podStartE2EDuration="4.029863902s" totalImagesPullingTime="2.045813039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:01.028665731 +0000 UTC m=+390.725786805" watchObservedRunningTime="2026-07-23 18:13:01.029863902 +0000 UTC m=+390.726985017" Jul 23 18:13:01.052215 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.052169 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" podStartSLOduration=1.589061394 podStartE2EDuration="3.052149524s" totalImagesPullingTime="1.46308813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:01.051982516 +0000 UTC m=+390.749103601" watchObservedRunningTime="2026-07-23 18:13:01.052149524 +0000 UTC m=+390.749270601" Jul 23 18:13:01.222930 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.222905 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" Jul 23 18:13:01.269309 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.269231 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md7nl\" (UniqueName: \"kubernetes.io/projected/3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff-kube-api-access-md7nl\") pod \"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff\" (UID: \"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff\") " Jul 23 18:13:01.271505 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.271478 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff-kube-api-access-md7nl" (OuterVolumeSpecName: "kube-api-access-md7nl") pod "3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff" (UID: "3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff"). InnerVolumeSpecName "kube-api-access-md7nl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:13:01.370823 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.370789 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md7nl\" (UniqueName: \"kubernetes.io/projected/3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff-kube-api-access-md7nl\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:01.985978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.985934 2555 generic.go:356] "Generic (PLEG): container finished" podID="3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff" containerID="c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8" exitCode=0 Jul 23 18:13:01.985978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.985979 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" Jul 23 18:13:01.986564 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.986029 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" event={"ID":"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff","Type":"ContainerDied","Data":"c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8"} Jul 23 18:13:01.986564 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.986091 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tfzzc" event={"ID":"3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff","Type":"ContainerDied","Data":"3f402b67011b6a95ce1be120ac7d29560b2a95a4b22e9a59fd304dc970bf9e1d"} Jul 23 18:13:01.986564 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.986123 2555 scope.go:117] "RemoveContainer" containerID="c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8" Jul 23 18:13:01.997833 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.996947 2555 scope.go:117] "RemoveContainer" containerID="c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8" Jul 23 18:13:01.998237 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:13:01.998112 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8\": container with ID starting with c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8 not found: ID does not exist" containerID="c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8" Jul 23 18:13:01.998237 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:01.998155 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8"} err="failed to get container status \"c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8\": rpc error: code = NotFound desc = could not find container \"c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8\": container with ID starting with c86590306137459f2adff616f16def6f484f73d2fc373b54287a48fa7db4d9d8 not found: ID does not exist" Jul 23 18:13:02.014922 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:02.014892 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tfzzc"] Jul 23 18:13:02.020082 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:02.020056 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tfzzc"] Jul 23 18:13:02.247332 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:02.247257 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff" path="/var/lib/kubelet/pods/3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff/volumes" Jul 23 18:13:08.522937 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:08.522895 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:13:08.523468 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:08.522967 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:13:08.545420 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:08.545394 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:13:09.046171 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:09.046141 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-fptmb" Jul 23 18:13:10.886096 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.886057 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-857c57d97c-t6m8v"] Jul 23 18:13:10.886502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.886423 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff" containerName="registry-server" Jul 23 18:13:10.886502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.886435 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff" containerName="registry-server" Jul 23 18:13:10.886502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.886486 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad08de5-e4a1-4a70-a2e5-ef4a535c30ff" containerName="registry-server" Jul 23 18:13:10.890885 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.890861 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.909252 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.909228 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857c57d97c-t6m8v"] Jul 23 18:13:10.954611 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.954569 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-serving-cert\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.954611 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.954612 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-service-ca\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.954815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.954694 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-oauth-config\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.954815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.954724 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-oauth-serving-cert\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.954815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.954775 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-trusted-ca-bundle\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.954815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.954798 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwns2\" (UniqueName: \"kubernetes.io/projected/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-kube-api-access-nwns2\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.954952 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.954845 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-config\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:10.963267 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:10.963245 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:11.045983 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.045947 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:11.055714 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.055684 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-trusted-ca-bundle\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.055856 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.055722 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwns2\" (UniqueName: \"kubernetes.io/projected/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-kube-api-access-nwns2\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.055856 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.055760 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-config\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.055856 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.055792 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-serving-cert\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.055856 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.055808 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-service-ca\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.055856 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.055825 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-oauth-config\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.055856 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.055839 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-oauth-serving-cert\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.056570 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.056543 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-oauth-serving-cert\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.056699 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.056663 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-service-ca\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.056699 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.056665 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-trusted-ca-bundle\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.056788 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.056730 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-config\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.058393 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.058338 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-serving-cert\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.058460 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.058389 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-console-oauth-config\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.065902 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.065878 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwns2\" (UniqueName: \"kubernetes.io/projected/05732ba1-fc4f-4b2c-b474-d5ce0e122f27-kube-api-access-nwns2\") pod \"console-857c57d97c-t6m8v\" (UID: \"05732ba1-fc4f-4b2c-b474-d5ce0e122f27\") " pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.199805 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.199710 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:11.335333 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:11.335257 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857c57d97c-t6m8v"] Jul 23 18:13:11.337775 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:13:11.337749 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05732ba1_fc4f_4b2c_b474_d5ce0e122f27.slice/crio-69466175b5daae78e6d6296597055479b7d83b71414d1b22cf648a2bfba2dd8c WatchSource:0}: Error finding container 69466175b5daae78e6d6296597055479b7d83b71414d1b22cf648a2bfba2dd8c: Status 404 returned error can't find the container with id 69466175b5daae78e6d6296597055479b7d83b71414d1b22cf648a2bfba2dd8c Jul 23 18:13:12.045018 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:12.044975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857c57d97c-t6m8v" event={"ID":"05732ba1-fc4f-4b2c-b474-d5ce0e122f27","Type":"ContainerStarted","Data":"7a3d5786e790fb332f6a8f791eaa750e93182a09932aead30db4348a2899b7df"} Jul 23 18:13:12.045437 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:12.045071 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857c57d97c-t6m8v" event={"ID":"05732ba1-fc4f-4b2c-b474-d5ce0e122f27","Type":"ContainerStarted","Data":"69466175b5daae78e6d6296597055479b7d83b71414d1b22cf648a2bfba2dd8c"} Jul 23 18:13:12.070894 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:12.070839 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-857c57d97c-t6m8v" podStartSLOduration=2.070822855 podStartE2EDuration="2.070822855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:12.067617802 +0000 UTC m=+401.764738907" watchObservedRunningTime="2026-07-23 18:13:12.070822855 +0000 UTC m=+401.767943929" Jul 23 18:13:18.786203 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:18.786164 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:21.200320 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:21.200273 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:21.200830 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:21.200354 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:21.205006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:21.204979 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:22.092529 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:22.092499 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-857c57d97c-t6m8v" Jul 23 18:13:22.150658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:22.150622 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6685644756-xwzns"] Jul 23 18:13:22.265837 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:22.265807 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:22.365568 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:22.365464 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:22.423565 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:22.423529 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:26.729593 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:26.729554 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:27.350500 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:27.350468 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:28.352594 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:28.352557 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:28.468072 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:28.468036 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:30.938002 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:30.937967 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw"] Jul 23 18:13:30.943156 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:30.943136 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" Jul 23 18:13:30.951308 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:30.951289 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-pk945\"" Jul 23 18:13:30.971181 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:30.971154 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw"] Jul 23 18:13:31.046096 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:31.046059 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djmtv\" (UniqueName: \"kubernetes.io/projected/fa9c2cdd-9897-4a1e-a5f7-b73cac95f856-kube-api-access-djmtv\") pod \"authorino-operator-75cc8d5cb7-kzhcw\" (UID: \"fa9c2cdd-9897-4a1e-a5f7-b73cac95f856\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" Jul 23 18:13:31.146978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:31.146939 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djmtv\" (UniqueName: \"kubernetes.io/projected/fa9c2cdd-9897-4a1e-a5f7-b73cac95f856-kube-api-access-djmtv\") pod \"authorino-operator-75cc8d5cb7-kzhcw\" (UID: \"fa9c2cdd-9897-4a1e-a5f7-b73cac95f856\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" Jul 23 18:13:31.168986 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:31.168958 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djmtv\" (UniqueName: \"kubernetes.io/projected/fa9c2cdd-9897-4a1e-a5f7-b73cac95f856-kube-api-access-djmtv\") pod \"authorino-operator-75cc8d5cb7-kzhcw\" (UID: \"fa9c2cdd-9897-4a1e-a5f7-b73cac95f856\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" Jul 23 18:13:31.253862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:31.253771 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" Jul 23 18:13:31.404813 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:31.404776 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw"] Jul 23 18:13:31.408149 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:13:31.408115 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9c2cdd_9897_4a1e_a5f7_b73cac95f856.slice/crio-d81afd9e2f2e823d4c5142e1b7f4eeda80aa7b35c434310721ebe53dabf1e595 WatchSource:0}: Error finding container d81afd9e2f2e823d4c5142e1b7f4eeda80aa7b35c434310721ebe53dabf1e595: Status 404 returned error can't find the container with id d81afd9e2f2e823d4c5142e1b7f4eeda80aa7b35c434310721ebe53dabf1e595 Jul 23 18:13:31.944094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:31.944059 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:32.139714 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:32.139679 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" event={"ID":"fa9c2cdd-9897-4a1e-a5f7-b73cac95f856","Type":"ContainerStarted","Data":"d81afd9e2f2e823d4c5142e1b7f4eeda80aa7b35c434310721ebe53dabf1e595"} Jul 23 18:13:33.626598 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.626557 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps"] Jul 23 18:13:33.630505 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.630472 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:13:33.633643 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.633616 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pl9zc\"" Jul 23 18:13:33.669563 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.669498 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps"] Jul 23 18:13:33.773158 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.773107 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpv4h\" (UniqueName: \"kubernetes.io/projected/f76858b9-0e68-4753-946a-f110d4718a21-kube-api-access-hpv4h\") pod \"limitador-operator-controller-manager-859d7b78bd-ndxps\" (UID: \"f76858b9-0e68-4753-946a-f110d4718a21\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:13:33.873902 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.873867 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpv4h\" (UniqueName: \"kubernetes.io/projected/f76858b9-0e68-4753-946a-f110d4718a21-kube-api-access-hpv4h\") pod \"limitador-operator-controller-manager-859d7b78bd-ndxps\" (UID: \"f76858b9-0e68-4753-946a-f110d4718a21\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:13:33.891558 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.891532 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpv4h\" (UniqueName: \"kubernetes.io/projected/f76858b9-0e68-4753-946a-f110d4718a21-kube-api-access-hpv4h\") pod \"limitador-operator-controller-manager-859d7b78bd-ndxps\" (UID: \"f76858b9-0e68-4753-946a-f110d4718a21\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:13:33.941851 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:33.941822 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:13:34.089936 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:34.089903 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps"] Jul 23 18:13:34.093162 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:13:34.093135 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76858b9_0e68_4753_946a_f110d4718a21.slice/crio-e5e322206af7286f361513a221616d08e40b7d60dbeb2cb529bd22adfa56e7da WatchSource:0}: Error finding container e5e322206af7286f361513a221616d08e40b7d60dbeb2cb529bd22adfa56e7da: Status 404 returned error can't find the container with id e5e322206af7286f361513a221616d08e40b7d60dbeb2cb529bd22adfa56e7da Jul 23 18:13:34.151538 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:34.151414 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" event={"ID":"fa9c2cdd-9897-4a1e-a5f7-b73cac95f856","Type":"ContainerStarted","Data":"56be43d568877ddd0e8a642e74f6395ce5d2b4eddbedd0d94a765563857d840d"} Jul 23 18:13:34.151538 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:34.151507 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" Jul 23 18:13:34.152891 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:34.152834 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" event={"ID":"f76858b9-0e68-4753-946a-f110d4718a21","Type":"ContainerStarted","Data":"e5e322206af7286f361513a221616d08e40b7d60dbeb2cb529bd22adfa56e7da"} Jul 23 18:13:34.178313 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:34.178255 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" podStartSLOduration=1.709524632 podStartE2EDuration="4.178237324s" totalImagesPullingTime="2.468712692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:34.17608714 +0000 UTC m=+423.873208233" watchObservedRunningTime="2026-07-23 18:13:34.178237324 +0000 UTC m=+423.875358398" Jul 23 18:13:34.329549 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:34.329514 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:34.944925 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:34.944883 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:35.537604 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:35.537569 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:36.038413 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.038359 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5"] Jul 23 18:13:36.041944 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.041919 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" Jul 23 18:13:36.045013 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.044990 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-s8gkm\"" Jul 23 18:13:36.045145 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.044990 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 23 18:13:36.055942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.055913 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5"] Jul 23 18:13:36.165802 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.165768 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" event={"ID":"f76858b9-0e68-4753-946a-f110d4718a21","Type":"ContainerStarted","Data":"010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168"} Jul 23 18:13:36.165980 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.165888 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:13:36.188113 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.188057 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" podStartSLOduration=1.607087846 podStartE2EDuration="3.188039337s" totalImagesPullingTime="1.580951491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:36.186256359 +0000 UTC m=+425.883377461" watchObservedRunningTime="2026-07-23 18:13:36.188039337 +0000 UTC m=+425.885160481" Jul 23 18:13:36.198008 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.197970 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzqcd\" (UniqueName: \"kubernetes.io/projected/4d232a16-8861-48a9-ad21-d98df7f36f62-kube-api-access-lzqcd\") pod \"dns-operator-controller-manager-864dd78c4c-ptrt5\" (UID: \"4d232a16-8861-48a9-ad21-d98df7f36f62\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" Jul 23 18:13:36.299462 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.299339 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzqcd\" (UniqueName: \"kubernetes.io/projected/4d232a16-8861-48a9-ad21-d98df7f36f62-kube-api-access-lzqcd\") pod \"dns-operator-controller-manager-864dd78c4c-ptrt5\" (UID: \"4d232a16-8861-48a9-ad21-d98df7f36f62\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" Jul 23 18:13:36.309885 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.309850 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzqcd\" (UniqueName: \"kubernetes.io/projected/4d232a16-8861-48a9-ad21-d98df7f36f62-kube-api-access-lzqcd\") pod \"dns-operator-controller-manager-864dd78c4c-ptrt5\" (UID: \"4d232a16-8861-48a9-ad21-d98df7f36f62\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" Jul 23 18:13:36.352978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.352934 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" Jul 23 18:13:36.493134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.493080 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5"] Jul 23 18:13:36.495656 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:13:36.495623 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d232a16_8861_48a9_ad21_d98df7f36f62.slice/crio-efc9a37bb8333f71b90edc27822256efd9be74cb63d5b692f551f1089d347433 WatchSource:0}: Error finding container efc9a37bb8333f71b90edc27822256efd9be74cb63d5b692f551f1089d347433: Status 404 returned error can't find the container with id efc9a37bb8333f71b90edc27822256efd9be74cb63d5b692f551f1089d347433 Jul 23 18:13:36.698680 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:36.698645 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:37.174150 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:37.174109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" event={"ID":"4d232a16-8861-48a9-ad21-d98df7f36f62","Type":"ContainerStarted","Data":"efc9a37bb8333f71b90edc27822256efd9be74cb63d5b692f551f1089d347433"} Jul 23 18:13:37.179221 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:37.179191 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:38.181639 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:38.181603 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" event={"ID":"4d232a16-8861-48a9-ad21-d98df7f36f62","Type":"ContainerStarted","Data":"96ae9c832a6871335fdb8e14a03579d43b57e5c78aff4381ee0a94406c9d4db8"} Jul 23 18:13:38.182052 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:38.181732 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" Jul 23 18:13:38.223328 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:38.223275 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" podStartSLOduration=0.700663902 podStartE2EDuration="2.22326072s" totalImagesPullingTime="1.522596818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:38.222491001 +0000 UTC m=+427.919612095" watchObservedRunningTime="2026-07-23 18:13:38.22326072 +0000 UTC m=+427.920381793" Jul 23 18:13:40.160174 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.160141 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:40.785158 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.785119 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r"] Jul 23 18:13:40.788811 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.788793 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:40.791530 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.791500 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qs55q\"" Jul 23 18:13:40.802202 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.802171 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r"] Jul 23 18:13:40.836933 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.836891 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz6wx\" (UniqueName: \"kubernetes.io/projected/2099d4f5-4cd7-4639-87cf-16874d7ccbca-kube-api-access-qz6wx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:40.837138 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.837004 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2099d4f5-4cd7-4639-87cf-16874d7ccbca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:40.937756 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.937716 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz6wx\" (UniqueName: \"kubernetes.io/projected/2099d4f5-4cd7-4639-87cf-16874d7ccbca-kube-api-access-qz6wx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:40.937947 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.937789 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2099d4f5-4cd7-4639-87cf-16874d7ccbca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:40.938168 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.938149 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2099d4f5-4cd7-4639-87cf-16874d7ccbca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:40.949104 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:40.949068 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz6wx\" (UniqueName: \"kubernetes.io/projected/2099d4f5-4cd7-4639-87cf-16874d7ccbca-kube-api-access-qz6wx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:41.099437 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:41.099401 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:41.251717 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:41.251589 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r"] Jul 23 18:13:41.254869 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:13:41.254835 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2099d4f5_4cd7_4639_87cf_16874d7ccbca.slice/crio-3f284d92058e0a1138a6fa95d556f3916fbedde638153d96d2953f3b645f238f WatchSource:0}: Error finding container 3f284d92058e0a1138a6fa95d556f3916fbedde638153d96d2953f3b645f238f: Status 404 returned error can't find the container with id 3f284d92058e0a1138a6fa95d556f3916fbedde638153d96d2953f3b645f238f Jul 23 18:13:42.006485 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:42.006450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:42.112677 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:42.112641 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:42.205751 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:42.205718 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" event={"ID":"2099d4f5-4cd7-4639-87cf-16874d7ccbca","Type":"ContainerStarted","Data":"3f284d92058e0a1138a6fa95d556f3916fbedde638153d96d2953f3b645f238f"} Jul 23 18:13:42.322289 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:42.322200 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:42.416840 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:42.416801 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:45.161646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:45.161604 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kzhcw" Jul 23 18:13:46.321252 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:46.321213 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:46.426083 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:46.426033 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:47.171847 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.171808 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6685644756-xwzns" podUID="6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" containerName="console" containerID="cri-o://c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725" gracePeriod=15 Jul 23 18:13:47.177334 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.177306 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:13:47.237605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.237562 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" event={"ID":"2099d4f5-4cd7-4639-87cf-16874d7ccbca","Type":"ContainerStarted","Data":"cd3e0288daadfb41787d2a6d5542828cf42a6e9295ced01a65eab1a55756bafa"} Jul 23 18:13:47.237756 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.237720 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:47.259695 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.259630 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" podStartSLOduration=2.116549715 podStartE2EDuration="7.259612379s" totalImagesPullingTime="5.143062664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:13:47.257446599 +0000 UTC m=+436.954567698" watchObservedRunningTime="2026-07-23 18:13:47.259612379 +0000 UTC m=+436.956733453" Jul 23 18:13:47.426281 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.426213 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:13:47.483766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.483722 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-serving-cert\") pod \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " Jul 23 18:13:47.483766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.483768 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-trusted-ca-bundle\") pod \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " Jul 23 18:13:47.484025 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.483801 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-oauth-serving-cert\") pod \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " Jul 23 18:13:47.484025 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.483841 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-config\") pod \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " Jul 23 18:13:47.484025 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484004 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-service-ca\") pod \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " Jul 23 18:13:47.484191 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484069 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4hzt\" (UniqueName: \"kubernetes.io/projected/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-kube-api-access-q4hzt\") pod \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " Jul 23 18:13:47.484191 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484103 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-oauth-config\") pod \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\" (UID: \"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6\") " Jul 23 18:13:47.484293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484232 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" (UID: "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:13:47.484293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484266 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" (UID: "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:13:47.484293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484284 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-config" (OuterVolumeSpecName: "console-config") pod "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" (UID: "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:13:47.484449 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484383 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" (UID: "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:13:47.484449 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484442 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-trusted-ca-bundle\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:47.484549 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484455 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-oauth-serving-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:47.484549 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484470 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-config\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:47.484549 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.484483 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:47.486002 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.485975 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" (UID: "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:13:47.486198 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.486172 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-kube-api-access-q4hzt" (OuterVolumeSpecName: "kube-api-access-q4hzt") pod "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" (UID: "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6"). InnerVolumeSpecName "kube-api-access-q4hzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:13:47.486262 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.486186 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" (UID: "6e1eef56-891b-4e63-8ddb-9dfbd9f470a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:13:47.585397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.585330 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4hzt\" (UniqueName: \"kubernetes.io/projected/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-kube-api-access-q4hzt\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:47.585397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.585390 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-oauth-config\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:47.585397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:47.585405 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6-console-serving-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:13:48.244428 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.244392 2555 generic.go:356] "Generic (PLEG): container finished" podID="6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" containerID="c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725" exitCode=0 Jul 23 18:13:48.244613 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.244514 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6685644756-xwzns" Jul 23 18:13:48.249053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.249013 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6685644756-xwzns" event={"ID":"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6","Type":"ContainerDied","Data":"c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725"} Jul 23 18:13:48.249216 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.249083 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6685644756-xwzns" event={"ID":"6e1eef56-891b-4e63-8ddb-9dfbd9f470a6","Type":"ContainerDied","Data":"9455074e421ed578abedaf7c7133259893b1fa33afee189e8d56f1c0ce0ef28e"} Jul 23 18:13:48.249216 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.249108 2555 scope.go:117] "RemoveContainer" containerID="c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725" Jul 23 18:13:48.258350 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.258328 2555 scope.go:117] "RemoveContainer" containerID="c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725" Jul 23 18:13:48.258734 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:13:48.258703 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725\": container with ID starting with c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725 not found: ID does not exist" containerID="c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725" Jul 23 18:13:48.258826 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.258747 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725"} err="failed to get container status \"c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725\": rpc error: code = NotFound desc = could not find container \"c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725\": container with ID starting with c89df09a16214f3793dae33b5d229d4018f1f267200e0b27d24f5f6a2f9de725 not found: ID does not exist" Jul 23 18:13:48.277866 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.277831 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6685644756-xwzns"] Jul 23 18:13:48.284219 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:48.284190 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6685644756-xwzns"] Jul 23 18:13:49.114066 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:49.114037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:49.189210 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:49.189179 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptrt5" Jul 23 18:13:49.212080 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:49.212039 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:49.309666 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:49.309629 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:49.797545 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:49.797499 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:50.247947 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:50.247909 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" path="/var/lib/kubelet/pods/6e1eef56-891b-4e63-8ddb-9dfbd9f470a6/volumes" Jul 23 18:13:53.683050 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:53.682969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:53.777638 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:53.777599 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:13:58.247524 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:58.247496 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:13:59.330420 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.330381 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc"] Jul 23 18:13:59.330810 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.330746 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" containerName="console" Jul 23 18:13:59.330810 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.330757 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" containerName="console" Jul 23 18:13:59.330883 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.330838 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1eef56-891b-4e63-8ddb-9dfbd9f470a6" containerName="console" Jul 23 18:13:59.333862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.333842 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.351344 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.351315 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc"] Jul 23 18:13:59.384641 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.384598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/988f2f30-81e3-432c-b2c1-489c1cc041f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.384825 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.384709 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85b65\" (UniqueName: \"kubernetes.io/projected/988f2f30-81e3-432c-b2c1-489c1cc041f7-kube-api-access-85b65\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.486030 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.485989 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85b65\" (UniqueName: \"kubernetes.io/projected/988f2f30-81e3-432c-b2c1-489c1cc041f7-kube-api-access-85b65\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.486236 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.486138 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/988f2f30-81e3-432c-b2c1-489c1cc041f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.486589 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.486570 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/988f2f30-81e3-432c-b2c1-489c1cc041f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.500528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.500495 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85b65\" (UniqueName: \"kubernetes.io/projected/988f2f30-81e3-432c-b2c1-489c1cc041f7-kube-api-access-85b65\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.643987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.643952 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:13:59.799005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:13:59.798981 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc"] Jul 23 18:13:59.801865 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:13:59.801827 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988f2f30_81e3_432c_b2c1_489c1cc041f7.slice/crio-119130793deedc56f9c698e339070f7180593eb27913a854f3cd96b1a4142e5f WatchSource:0}: Error finding container 119130793deedc56f9c698e339070f7180593eb27913a854f3cd96b1a4142e5f: Status 404 returned error can't find the container with id 119130793deedc56f9c698e339070f7180593eb27913a854f3cd96b1a4142e5f Jul 23 18:14:00.114228 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.114185 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc"] Jul 23 18:14:00.128239 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.128206 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc"] Jul 23 18:14:00.148183 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.148150 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r"] Jul 23 18:14:00.148427 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.148360 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" podUID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" containerName="manager" containerID="cri-o://cd3e0288daadfb41787d2a6d5542828cf42a6e9295ced01a65eab1a55756bafa" gracePeriod=2 Jul 23 18:14:00.153535 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.153505 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj"] Jul 23 18:14:00.154076 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.154058 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" containerName="manager" Jul 23 18:14:00.154124 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.154083 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" containerName="manager" Jul 23 18:14:00.154247 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.154234 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" containerName="manager" Jul 23 18:14:00.215858 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.215816 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r"] Jul 23 18:14:00.216026 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.215868 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj"] Jul 23 18:14:00.216026 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.215897 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps"] Jul 23 18:14:00.216026 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.215929 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457"] Jul 23 18:14:00.216464 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.216443 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.217192 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.217152 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" podUID="f76858b9-0e68-4753-946a-f110d4718a21" containerName="manager" containerID="cri-o://010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168" gracePeriod=2 Jul 23 18:14:00.217812 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.216513 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" containerName="manager" Jul 23 18:14:00.217812 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.217614 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" containerName="manager" Jul 23 18:14:00.217812 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.217752 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" containerName="manager" Jul 23 18:14:00.219750 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:00.219690 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ndxps\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="f76858b9-0e68-4753-946a-f110d4718a21" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:00.222215 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.222283 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps"] Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.222325 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457"] Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.222354 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf"] Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.222358 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.222918 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f76858b9-0e68-4753-946a-f110d4718a21" containerName="manager" Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.222935 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f76858b9-0e68-4753-946a-f110d4718a21" containerName="manager" Jul 23 18:14:00.223504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.223038 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f76858b9-0e68-4753-946a-f110d4718a21" containerName="manager" Jul 23 18:14:00.224646 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:00.224513 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ndxps\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="f76858b9-0e68-4753-946a-f110d4718a21" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:14:00.226643 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.226623 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" Jul 23 18:14:00.226796 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.226776 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf"] Jul 23 18:14:00.252625 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:00.252576 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nps9r\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:14:00.291815 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.291777 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84aba275-c185-4380-88cc-0d521ed06a65-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qn457\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.292029 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.291835 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-996vc\" (UniqueName: \"kubernetes.io/projected/84aba275-c185-4380-88cc-0d521ed06a65-kube-api-access-996vc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qn457\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.292029 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.291963 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p92t4\" (UniqueName: \"kubernetes.io/projected/e9d85b29-b883-44ee-8e76-b6e465091b46-kube-api-access-p92t4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt5kj\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.292029 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.292002 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6zt2\" (UniqueName: \"kubernetes.io/projected/35e26d70-e755-4fc6-b8f8-1e739c482867-kube-api-access-x6zt2\") pod \"limitador-operator-controller-manager-859d7b78bd-v7zvf\" (UID: \"35e26d70-e755-4fc6-b8f8-1e739c482867\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" Jul 23 18:14:00.292151 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.292036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d85b29-b883-44ee-8e76-b6e465091b46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt5kj\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.307089 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:00.307058 2555 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" container="manager" Jul 23 18:14:00.308694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.308663 2555 generic.go:356] "Generic (PLEG): container finished" podID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" containerID="cd3e0288daadfb41787d2a6d5542828cf42a6e9295ced01a65eab1a55756bafa" exitCode=0 Jul 23 18:14:00.393014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.392960 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6zt2\" (UniqueName: \"kubernetes.io/projected/35e26d70-e755-4fc6-b8f8-1e739c482867-kube-api-access-x6zt2\") pod \"limitador-operator-controller-manager-859d7b78bd-v7zvf\" (UID: \"35e26d70-e755-4fc6-b8f8-1e739c482867\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" Jul 23 18:14:00.393440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.393012 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d85b29-b883-44ee-8e76-b6e465091b46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt5kj\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.393440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.393072 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84aba275-c185-4380-88cc-0d521ed06a65-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qn457\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.393440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.393118 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-996vc\" (UniqueName: \"kubernetes.io/projected/84aba275-c185-4380-88cc-0d521ed06a65-kube-api-access-996vc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qn457\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.393440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.393194 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p92t4\" (UniqueName: \"kubernetes.io/projected/e9d85b29-b883-44ee-8e76-b6e465091b46-kube-api-access-p92t4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt5kj\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.393875 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.393800 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84aba275-c185-4380-88cc-0d521ed06a65-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qn457\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.394166 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.394118 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d85b29-b883-44ee-8e76-b6e465091b46-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt5kj\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.407961 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.407931 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-996vc\" (UniqueName: \"kubernetes.io/projected/84aba275-c185-4380-88cc-0d521ed06a65-kube-api-access-996vc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qn457\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.408357 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.408329 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p92t4\" (UniqueName: \"kubernetes.io/projected/e9d85b29-b883-44ee-8e76-b6e465091b46-kube-api-access-p92t4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt5kj\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.408961 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.408940 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6zt2\" (UniqueName: \"kubernetes.io/projected/35e26d70-e755-4fc6-b8f8-1e739c482867-kube-api-access-x6zt2\") pod \"limitador-operator-controller-manager-859d7b78bd-v7zvf\" (UID: \"35e26d70-e755-4fc6-b8f8-1e739c482867\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" Jul 23 18:14:00.457610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.457586 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:14:00.464845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.464713 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:14:00.493925 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.493891 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpv4h\" (UniqueName: \"kubernetes.io/projected/f76858b9-0e68-4753-946a-f110d4718a21-kube-api-access-hpv4h\") pod \"f76858b9-0e68-4753-946a-f110d4718a21\" (UID: \"f76858b9-0e68-4753-946a-f110d4718a21\") " Jul 23 18:14:00.494114 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.493967 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2099d4f5-4cd7-4639-87cf-16874d7ccbca-extensions-socket-volume\") pod \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " Jul 23 18:14:00.494114 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.494087 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz6wx\" (UniqueName: \"kubernetes.io/projected/2099d4f5-4cd7-4639-87cf-16874d7ccbca-kube-api-access-qz6wx\") pod \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\" (UID: \"2099d4f5-4cd7-4639-87cf-16874d7ccbca\") " Jul 23 18:14:00.494532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.494503 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2099d4f5-4cd7-4639-87cf-16874d7ccbca-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2099d4f5-4cd7-4639-87cf-16874d7ccbca" (UID: "2099d4f5-4cd7-4639-87cf-16874d7ccbca"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:14:00.496056 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.496034 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f76858b9-0e68-4753-946a-f110d4718a21-kube-api-access-hpv4h" (OuterVolumeSpecName: "kube-api-access-hpv4h") pod "f76858b9-0e68-4753-946a-f110d4718a21" (UID: "f76858b9-0e68-4753-946a-f110d4718a21"). InnerVolumeSpecName "kube-api-access-hpv4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:14:00.496104 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.496068 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2099d4f5-4cd7-4639-87cf-16874d7ccbca-kube-api-access-qz6wx" (OuterVolumeSpecName: "kube-api-access-qz6wx") pod "2099d4f5-4cd7-4639-87cf-16874d7ccbca" (UID: "2099d4f5-4cd7-4639-87cf-16874d7ccbca"). InnerVolumeSpecName "kube-api-access-qz6wx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:14:00.595471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.595433 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz6wx\" (UniqueName: \"kubernetes.io/projected/2099d4f5-4cd7-4639-87cf-16874d7ccbca-kube-api-access-qz6wx\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:00.595471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.595465 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpv4h\" (UniqueName: \"kubernetes.io/projected/f76858b9-0e68-4753-946a-f110d4718a21-kube-api-access-hpv4h\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:00.595471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.595480 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2099d4f5-4cd7-4639-87cf-16874d7ccbca-extensions-socket-volume\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:00.624149 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.624118 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:00.636287 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.636252 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:00.645312 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.645244 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" Jul 23 18:14:00.794192 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:00.794038 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj"] Jul 23 18:14:00.796888 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:14:00.796865 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d85b29_b883_44ee_8e76_b6e465091b46.slice/crio-306a3c29df92498ed4bc6f850aa99ce6b78d6410a4127797d11af076fcd9d49e WatchSource:0}: Error finding container 306a3c29df92498ed4bc6f850aa99ce6b78d6410a4127797d11af076fcd9d49e: Status 404 returned error can't find the container with id 306a3c29df92498ed4bc6f850aa99ce6b78d6410a4127797d11af076fcd9d49e Jul 23 18:14:01.022444 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.022412 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf"] Jul 23 18:14:01.023854 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:14:01.023816 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e26d70_e755_4fc6_b8f8_1e739c482867.slice/crio-ffaf41c461eaf4b1cf19917c65c539fecdbb1e93c268ee642990f05389e54e63 WatchSource:0}: Error finding container ffaf41c461eaf4b1cf19917c65c539fecdbb1e93c268ee642990f05389e54e63: Status 404 returned error can't find the container with id ffaf41c461eaf4b1cf19917c65c539fecdbb1e93c268ee642990f05389e54e63 Jul 23 18:14:01.024074 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.024048 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457"] Jul 23 18:14:01.024316 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:14:01.024296 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84aba275_c185_4380_88cc_0d521ed06a65.slice/crio-5d03ca26eb0905e4608de0ea8bff57c463828c65e364f1bd7359512a92caf935 WatchSource:0}: Error finding container 5d03ca26eb0905e4608de0ea8bff57c463828c65e364f1bd7359512a92caf935: Status 404 returned error can't find the container with id 5d03ca26eb0905e4608de0ea8bff57c463828c65e364f1bd7359512a92caf935 Jul 23 18:14:01.317974 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.317883 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" containerName="manager" containerID="cri-o://1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e" gracePeriod=2 Jul 23 18:14:01.319428 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.319404 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nps9r" Jul 23 18:14:01.319428 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.319419 2555 scope.go:117] "RemoveContainer" containerID="cd3e0288daadfb41787d2a6d5542828cf42a6e9295ced01a65eab1a55756bafa" Jul 23 18:14:01.320508 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:01.320477 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:01.321318 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.321189 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" event={"ID":"35e26d70-e755-4fc6-b8f8-1e739c482867","Type":"ContainerStarted","Data":"d18bc9a9f24a09988b47ce90b0a857f470cf46298472dc84995cdf4cfb7723e2"} Jul 23 18:14:01.321318 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.321241 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" Jul 23 18:14:01.321318 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.321264 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" event={"ID":"35e26d70-e755-4fc6-b8f8-1e739c482867","Type":"ContainerStarted","Data":"ffaf41c461eaf4b1cf19917c65c539fecdbb1e93c268ee642990f05389e54e63"} Jul 23 18:14:01.323052 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.323012 2555 generic.go:356] "Generic (PLEG): container finished" podID="f76858b9-0e68-4753-946a-f110d4718a21" containerID="010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168" exitCode=0 Jul 23 18:14:01.323324 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:01.323147 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:01.323612 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.323214 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ndxps" Jul 23 18:14:01.325412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.325334 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" event={"ID":"84aba275-c185-4380-88cc-0d521ed06a65","Type":"ContainerStarted","Data":"7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d"} Jul 23 18:14:01.325502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.325424 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:01.325502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.325452 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" event={"ID":"84aba275-c185-4380-88cc-0d521ed06a65","Type":"ContainerStarted","Data":"5d03ca26eb0905e4608de0ea8bff57c463828c65e364f1bd7359512a92caf935"} Jul 23 18:14:01.327384 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.327348 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" event={"ID":"e9d85b29-b883-44ee-8e76-b6e465091b46","Type":"ContainerStarted","Data":"cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8"} Jul 23 18:14:01.327467 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.327408 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" event={"ID":"e9d85b29-b883-44ee-8e76-b6e465091b46","Type":"ContainerStarted","Data":"306a3c29df92498ed4bc6f850aa99ce6b78d6410a4127797d11af076fcd9d49e"} Jul 23 18:14:01.327533 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.327515 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:01.331786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.331671 2555 scope.go:117] "RemoveContainer" containerID="010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168" Jul 23 18:14:01.362608 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:01.362572 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:01.362826 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.362795 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" podStartSLOduration=1.362783107 podStartE2EDuration="1.362783107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:14:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:14:01.360079608 +0000 UTC m=+451.057200682" watchObservedRunningTime="2026-07-23 18:14:01.362783107 +0000 UTC m=+451.059904181" Jul 23 18:14:01.390776 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.390719 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" podStartSLOduration=1.390706667 podStartE2EDuration="1.390706667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:14:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:14:01.387999155 +0000 UTC m=+451.085120259" watchObservedRunningTime="2026-07-23 18:14:01.390706667 +0000 UTC m=+451.087827741" Jul 23 18:14:01.391261 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:01.391236 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:01.431294 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.431274 2555 scope.go:117] "RemoveContainer" containerID="010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168" Jul 23 18:14:01.431726 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.431583 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" podStartSLOduration=1.431563547 podStartE2EDuration="1.431563547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:14:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:14:01.429917985 +0000 UTC m=+451.127039087" watchObservedRunningTime="2026-07-23 18:14:01.431563547 +0000 UTC m=+451.128684622" Jul 23 18:14:01.431726 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:01.431623 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168\": container with ID starting with 010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168 not found: ID does not exist" containerID="010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168" Jul 23 18:14:01.431726 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.431652 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168"} err="failed to get container status \"010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168\": rpc error: code = NotFound desc = could not find container \"010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168\": container with ID starting with 010f1b50426785b10de66db60bb78e69686aab06ebb27ecf56b453c53a9d8168 not found: ID does not exist" Jul 23 18:14:01.433494 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:01.433413 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:01.567506 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.567474 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:01.570156 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:01.570109 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:01.603575 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.603548 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85b65\" (UniqueName: \"kubernetes.io/projected/988f2f30-81e3-432c-b2c1-489c1cc041f7-kube-api-access-85b65\") pod \"988f2f30-81e3-432c-b2c1-489c1cc041f7\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " Jul 23 18:14:01.603665 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.603641 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/988f2f30-81e3-432c-b2c1-489c1cc041f7-extensions-socket-volume\") pod \"988f2f30-81e3-432c-b2c1-489c1cc041f7\" (UID: \"988f2f30-81e3-432c-b2c1-489c1cc041f7\") " Jul 23 18:14:01.604012 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.603990 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988f2f30-81e3-432c-b2c1-489c1cc041f7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "988f2f30-81e3-432c-b2c1-489c1cc041f7" (UID: "988f2f30-81e3-432c-b2c1-489c1cc041f7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:14:01.605558 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.605531 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988f2f30-81e3-432c-b2c1-489c1cc041f7-kube-api-access-85b65" (OuterVolumeSpecName: "kube-api-access-85b65") pod "988f2f30-81e3-432c-b2c1-489c1cc041f7" (UID: "988f2f30-81e3-432c-b2c1-489c1cc041f7"). InnerVolumeSpecName "kube-api-access-85b65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:14:01.704797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.704764 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85b65\" (UniqueName: \"kubernetes.io/projected/988f2f30-81e3-432c-b2c1-489c1cc041f7-kube-api-access-85b65\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:01.704797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.704793 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/988f2f30-81e3-432c-b2c1-489c1cc041f7-extensions-socket-volume\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:01.796263 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.796230 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:01.889409 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:01.889356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:02.246890 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:02.246801 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:02.248465 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.248436 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2099d4f5-4cd7-4639-87cf-16874d7ccbca" path="/var/lib/kubelet/pods/2099d4f5-4cd7-4639-87cf-16874d7ccbca/volumes" Jul 23 18:14:02.248889 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.248871 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" path="/var/lib/kubelet/pods/988f2f30-81e3-432c-b2c1-489c1cc041f7/volumes" Jul 23 18:14:02.249222 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.249205 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f76858b9-0e68-4753-946a-f110d4718a21" path="/var/lib/kubelet/pods/f76858b9-0e68-4753-946a-f110d4718a21/volumes" Jul 23 18:14:02.335680 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.335649 2555 generic.go:356] "Generic (PLEG): container finished" podID="988f2f30-81e3-432c-b2c1-489c1cc041f7" containerID="1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e" exitCode=0 Jul 23 18:14:02.335825 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.335693 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:02.335825 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.335767 2555 scope.go:117] "RemoveContainer" containerID="1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e" Jul 23 18:14:02.340904 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:02.340873 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-flplc\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="988f2f30-81e3-432c-b2c1-489c1cc041f7" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-flplc" Jul 23 18:14:02.344248 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.344235 2555 scope.go:117] "RemoveContainer" containerID="1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e" Jul 23 18:14:02.344513 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:02.344494 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e\": container with ID starting with 1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e not found: ID does not exist" containerID="1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e" Jul 23 18:14:02.344569 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.344519 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e"} err="failed to get container status \"1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e\": rpc error: code = NotFound desc = could not find container \"1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e\": container with ID starting with 1a6c979bf95dbb634b994e7f48de7c8b6bfb4c0096cf3881402e45673363a90e not found: ID does not exist" Jul 23 18:14:02.473097 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.473063 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:02.571099 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:02.571019 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:06.125179 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:06.125146 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:06.227768 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:06.227738 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:08.351871 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:08.351833 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:08.411945 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:08.411912 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:12.338071 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.338038 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:12.338515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.338111 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v7zvf" Jul 23 18:14:12.338515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.338272 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:12.434527 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.434494 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457"] Jul 23 18:14:12.434740 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.434713 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" podUID="84aba275-c185-4380-88cc-0d521ed06a65" containerName="manager" containerID="cri-o://7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d" gracePeriod=10 Jul 23 18:14:12.695423 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.695394 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:12.771792 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.771755 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx"] Jul 23 18:14:12.772144 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.772132 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84aba275-c185-4380-88cc-0d521ed06a65" containerName="manager" Jul 23 18:14:12.772187 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.772146 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="84aba275-c185-4380-88cc-0d521ed06a65" containerName="manager" Jul 23 18:14:12.772223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.772219 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="84aba275-c185-4380-88cc-0d521ed06a65" containerName="manager" Jul 23 18:14:12.775486 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.775469 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:12.792537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.792496 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx"] Jul 23 18:14:12.801651 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.801620 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84aba275-c185-4380-88cc-0d521ed06a65-extensions-socket-volume\") pod \"84aba275-c185-4380-88cc-0d521ed06a65\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " Jul 23 18:14:12.801797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.801683 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-996vc\" (UniqueName: \"kubernetes.io/projected/84aba275-c185-4380-88cc-0d521ed06a65-kube-api-access-996vc\") pod \"84aba275-c185-4380-88cc-0d521ed06a65\" (UID: \"84aba275-c185-4380-88cc-0d521ed06a65\") " Jul 23 18:14:12.802110 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.802081 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84aba275-c185-4380-88cc-0d521ed06a65-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "84aba275-c185-4380-88cc-0d521ed06a65" (UID: "84aba275-c185-4380-88cc-0d521ed06a65"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:14:12.803838 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.803815 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84aba275-c185-4380-88cc-0d521ed06a65-kube-api-access-996vc" (OuterVolumeSpecName: "kube-api-access-996vc") pod "84aba275-c185-4380-88cc-0d521ed06a65" (UID: "84aba275-c185-4380-88cc-0d521ed06a65"). InnerVolumeSpecName "kube-api-access-996vc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:14:12.902504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.902407 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m74lz\" (UniqueName: \"kubernetes.io/projected/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-kube-api-access-m74lz\") pod \"kuadrant-operator-controller-manager-86878c769f-xgpjx\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:12.902667 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.902510 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xgpjx\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:12.902667 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.902594 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/84aba275-c185-4380-88cc-0d521ed06a65-extensions-socket-volume\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:12.902667 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:12.902605 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-996vc\" (UniqueName: \"kubernetes.io/projected/84aba275-c185-4380-88cc-0d521ed06a65-kube-api-access-996vc\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:13.003273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.003233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xgpjx\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:13.003474 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.003321 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m74lz\" (UniqueName: \"kubernetes.io/projected/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-kube-api-access-m74lz\") pod \"kuadrant-operator-controller-manager-86878c769f-xgpjx\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:13.003656 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.003634 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xgpjx\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:13.023069 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.023035 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m74lz\" (UniqueName: \"kubernetes.io/projected/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-kube-api-access-m74lz\") pod \"kuadrant-operator-controller-manager-86878c769f-xgpjx\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:13.085565 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.085532 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:13.244355 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.244178 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx"] Jul 23 18:14:13.247417 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:14:13.247389 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4156e3da_0483_48f2_8a3d_8b4c0b23ee22.slice/crio-5544a373edee50501401edaee97fefe15bc11b13f56f5b92b7017d183e896728 WatchSource:0}: Error finding container 5544a373edee50501401edaee97fefe15bc11b13f56f5b92b7017d183e896728: Status 404 returned error can't find the container with id 5544a373edee50501401edaee97fefe15bc11b13f56f5b92b7017d183e896728 Jul 23 18:14:13.398661 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.398624 2555 generic.go:356] "Generic (PLEG): container finished" podID="84aba275-c185-4380-88cc-0d521ed06a65" containerID="7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d" exitCode=0 Jul 23 18:14:13.399098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.398666 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" Jul 23 18:14:13.399098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.398706 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" event={"ID":"84aba275-c185-4380-88cc-0d521ed06a65","Type":"ContainerDied","Data":"7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d"} Jul 23 18:14:13.399098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.398766 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457" event={"ID":"84aba275-c185-4380-88cc-0d521ed06a65","Type":"ContainerDied","Data":"5d03ca26eb0905e4608de0ea8bff57c463828c65e364f1bd7359512a92caf935"} Jul 23 18:14:13.399098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.398793 2555 scope.go:117] "RemoveContainer" containerID="7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d" Jul 23 18:14:13.400422 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.400396 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" event={"ID":"4156e3da-0483-48f2-8a3d-8b4c0b23ee22","Type":"ContainerStarted","Data":"3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad"} Jul 23 18:14:13.400543 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.400448 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" event={"ID":"4156e3da-0483-48f2-8a3d-8b4c0b23ee22","Type":"ContainerStarted","Data":"5544a373edee50501401edaee97fefe15bc11b13f56f5b92b7017d183e896728"} Jul 23 18:14:13.400607 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.400553 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:13.414892 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.414866 2555 scope.go:117] "RemoveContainer" containerID="7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d" Jul 23 18:14:13.415245 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:13.415219 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d\": container with ID starting with 7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d not found: ID does not exist" containerID="7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d" Jul 23 18:14:13.415332 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.415258 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d"} err="failed to get container status \"7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d\": rpc error: code = NotFound desc = could not find container \"7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d\": container with ID starting with 7305b14aad25ac80c3fd50ab5537150fde399eb5193213cefae19ef92e883a3d not found: ID does not exist" Jul 23 18:14:13.428743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.428648 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" podStartSLOduration=1.428632478 podStartE2EDuration="1.428632478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:14:13.426867589 +0000 UTC m=+463.123988681" watchObservedRunningTime="2026-07-23 18:14:13.428632478 +0000 UTC m=+463.125753552" Jul 23 18:14:13.448046 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.448011 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457"] Jul 23 18:14:13.457055 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.457021 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qn457"] Jul 23 18:14:13.754598 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.754519 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:13.854600 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:13.854564 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:14.248067 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:14.248031 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84aba275-c185-4380-88cc-0d521ed06a65" path="/var/lib/kubelet/pods/84aba275-c185-4380-88cc-0d521ed06a65/volumes" Jul 23 18:14:24.408617 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.408573 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:14:24.467743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.467706 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj"] Jul 23 18:14:24.467983 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.467955 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" podUID="e9d85b29-b883-44ee-8e76-b6e465091b46" containerName="manager" containerID="cri-o://cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8" gracePeriod=10 Jul 23 18:14:24.731190 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.731159 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:24.812880 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.812821 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p92t4\" (UniqueName: \"kubernetes.io/projected/e9d85b29-b883-44ee-8e76-b6e465091b46-kube-api-access-p92t4\") pod \"e9d85b29-b883-44ee-8e76-b6e465091b46\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " Jul 23 18:14:24.813074 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.812901 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d85b29-b883-44ee-8e76-b6e465091b46-extensions-socket-volume\") pod \"e9d85b29-b883-44ee-8e76-b6e465091b46\" (UID: \"e9d85b29-b883-44ee-8e76-b6e465091b46\") " Jul 23 18:14:24.813349 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.813320 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9d85b29-b883-44ee-8e76-b6e465091b46-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e9d85b29-b883-44ee-8e76-b6e465091b46" (UID: "e9d85b29-b883-44ee-8e76-b6e465091b46"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:14:24.815101 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.815078 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9d85b29-b883-44ee-8e76-b6e465091b46-kube-api-access-p92t4" (OuterVolumeSpecName: "kube-api-access-p92t4") pod "e9d85b29-b883-44ee-8e76-b6e465091b46" (UID: "e9d85b29-b883-44ee-8e76-b6e465091b46"). InnerVolumeSpecName "kube-api-access-p92t4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:14:24.914326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.914288 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p92t4\" (UniqueName: \"kubernetes.io/projected/e9d85b29-b883-44ee-8e76-b6e465091b46-kube-api-access-p92t4\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:24.914326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:24.914322 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9d85b29-b883-44ee-8e76-b6e465091b46-extensions-socket-volume\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:25.467152 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.467109 2555 generic.go:356] "Generic (PLEG): container finished" podID="e9d85b29-b883-44ee-8e76-b6e465091b46" containerID="cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8" exitCode=0 Jul 23 18:14:25.467694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.467159 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" Jul 23 18:14:25.467694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.467199 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" event={"ID":"e9d85b29-b883-44ee-8e76-b6e465091b46","Type":"ContainerDied","Data":"cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8"} Jul 23 18:14:25.467694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.467255 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj" event={"ID":"e9d85b29-b883-44ee-8e76-b6e465091b46","Type":"ContainerDied","Data":"306a3c29df92498ed4bc6f850aa99ce6b78d6410a4127797d11af076fcd9d49e"} Jul 23 18:14:25.467694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.467282 2555 scope.go:117] "RemoveContainer" containerID="cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8" Jul 23 18:14:25.476546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.476321 2555 scope.go:117] "RemoveContainer" containerID="cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8" Jul 23 18:14:25.476703 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:25.476684 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8\": container with ID starting with cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8 not found: ID does not exist" containerID="cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8" Jul 23 18:14:25.476752 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.476712 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8"} err="failed to get container status \"cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8\": rpc error: code = NotFound desc = could not find container \"cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8\": container with ID starting with cf627ba30b135c89da4e63bee187866fcc481834ac88fe662a8382e05248abb8 not found: ID does not exist" Jul 23 18:14:25.499810 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.499775 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj"] Jul 23 18:14:25.501284 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:25.501263 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt5kj"] Jul 23 18:14:26.248316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:26.248277 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9d85b29-b883-44ee-8e76-b6e465091b46" path="/var/lib/kubelet/pods/e9d85b29-b883-44ee-8e76-b6e465091b46/volumes" Jul 23 18:14:28.850094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:28.850056 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:29.175739 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:29.175654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:31.959024 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:31.958991 2555 scope.go:117] "RemoveContainer" containerID="a029b60bbca1b472a017bbc18b21d766a4d00ececc21729c3e8ab4c985c23143" Jul 23 18:14:39.129733 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:39.129703 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:39.230515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:39.230482 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:41.402479 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.402439 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:41.412183 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.412153 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-78t9v"] Jul 23 18:14:41.412596 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.412583 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d85b29-b883-44ee-8e76-b6e465091b46" containerName="manager" Jul 23 18:14:41.412667 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.412609 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d85b29-b883-44ee-8e76-b6e465091b46" containerName="manager" Jul 23 18:14:41.412703 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.412698 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d85b29-b883-44ee-8e76-b6e465091b46" containerName="manager" Jul 23 18:14:41.417475 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.417459 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.419987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.419966 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 18:14:41.420047 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.419987 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k9zht\"" Jul 23 18:14:41.423719 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.423697 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-78t9v"] Jul 23 18:14:41.454511 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.454479 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b6ca3609-1396-43d0-a027-1e513a686091-config-file\") pod \"limitador-limitador-64c94fd64f-78t9v\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.454696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.454530 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbbmt\" (UniqueName: \"kubernetes.io/projected/b6ca3609-1396-43d0-a027-1e513a686091-kube-api-access-dbbmt\") pod \"limitador-limitador-64c94fd64f-78t9v\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.508382 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.508330 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-78t9v"] Jul 23 18:14:41.525136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.525104 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:41.555679 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.555650 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b6ca3609-1396-43d0-a027-1e513a686091-config-file\") pod \"limitador-limitador-64c94fd64f-78t9v\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.555884 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.555699 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbbmt\" (UniqueName: \"kubernetes.io/projected/b6ca3609-1396-43d0-a027-1e513a686091-kube-api-access-dbbmt\") pod \"limitador-limitador-64c94fd64f-78t9v\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.556318 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.556292 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b6ca3609-1396-43d0-a027-1e513a686091-config-file\") pod \"limitador-limitador-64c94fd64f-78t9v\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.566383 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.566340 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbbmt\" (UniqueName: \"kubernetes.io/projected/b6ca3609-1396-43d0-a027-1e513a686091-kube-api-access-dbbmt\") pod \"limitador-limitador-64c94fd64f-78t9v\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.625647 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.625613 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:41.729148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.729052 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:41.872977 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:41.872944 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-78t9v"] Jul 23 18:14:41.876488 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:14:41.876453 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ca3609_1396_43d0_a027_1e513a686091.slice/crio-d2ff430063ecf84ed3dfdf7c0fd33a3c41e7740c97c86821353c1a1423accf69 WatchSource:0}: Error finding container d2ff430063ecf84ed3dfdf7c0fd33a3c41e7740c97c86821353c1a1423accf69: Status 404 returned error can't find the container with id d2ff430063ecf84ed3dfdf7c0fd33a3c41e7740c97c86821353c1a1423accf69 Jul 23 18:14:42.293881 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.293835 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vxtwl"] Jul 23 18:14:42.299072 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.299047 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vxtwl" Jul 23 18:14:42.301504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.301481 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hxqdz\"" Jul 23 18:14:42.305728 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.305705 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vxtwl"] Jul 23 18:14:42.339680 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.339649 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:42.363049 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.363009 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr4w5\" (UniqueName: \"kubernetes.io/projected/7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b-kube-api-access-lr4w5\") pod \"authorino-58cd955675-vxtwl\" (UID: \"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b\") " pod="kuadrant-system/authorino-58cd955675-vxtwl" Jul 23 18:14:42.463670 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.463628 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr4w5\" (UniqueName: \"kubernetes.io/projected/7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b-kube-api-access-lr4w5\") pod \"authorino-58cd955675-vxtwl\" (UID: \"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b\") " pod="kuadrant-system/authorino-58cd955675-vxtwl" Jul 23 18:14:42.473557 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.473518 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr4w5\" (UniqueName: \"kubernetes.io/projected/7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b-kube-api-access-lr4w5\") pod \"authorino-58cd955675-vxtwl\" (UID: \"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b\") " pod="kuadrant-system/authorino-58cd955675-vxtwl" Jul 23 18:14:42.526272 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.526237 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:14:42.551499 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.551405 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hd24d"] Jul 23 18:14:42.555259 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.555229 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hd24d" Jul 23 18:14:42.562777 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.562740 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hd24d"] Jul 23 18:14:42.563816 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.563783 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" event={"ID":"b6ca3609-1396-43d0-a027-1e513a686091","Type":"ContainerStarted","Data":"d2ff430063ecf84ed3dfdf7c0fd33a3c41e7740c97c86821353c1a1423accf69"} Jul 23 18:14:42.564294 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.564269 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5msv6\" (UniqueName: \"kubernetes.io/projected/41a242aa-7d7d-42ac-808a-e10fdcf23856-kube-api-access-5msv6\") pod \"authorino-75d556bd77-hd24d\" (UID: \"41a242aa-7d7d-42ac-808a-e10fdcf23856\") " pod="kuadrant-system/authorino-75d556bd77-hd24d" Jul 23 18:14:42.609964 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.609772 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vxtwl" Jul 23 18:14:42.665581 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.665541 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5msv6\" (UniqueName: \"kubernetes.io/projected/41a242aa-7d7d-42ac-808a-e10fdcf23856-kube-api-access-5msv6\") pod \"authorino-75d556bd77-hd24d\" (UID: \"41a242aa-7d7d-42ac-808a-e10fdcf23856\") " pod="kuadrant-system/authorino-75d556bd77-hd24d" Jul 23 18:14:42.676446 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.676359 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5msv6\" (UniqueName: \"kubernetes.io/projected/41a242aa-7d7d-42ac-808a-e10fdcf23856-kube-api-access-5msv6\") pod \"authorino-75d556bd77-hd24d\" (UID: \"41a242aa-7d7d-42ac-808a-e10fdcf23856\") " pod="kuadrant-system/authorino-75d556bd77-hd24d" Jul 23 18:14:42.824492 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.824200 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vxtwl"] Jul 23 18:14:42.829318 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:14:42.829246 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a0ebbc4_d60d_4e8a_9940_ac1e95932d2b.slice/crio-e8cdc664d707d5794ea0ebd123c7d50fe155c4a7c01be5bafdd10f33623ab53c WatchSource:0}: Error finding container e8cdc664d707d5794ea0ebd123c7d50fe155c4a7c01be5bafdd10f33623ab53c: Status 404 returned error can't find the container with id e8cdc664d707d5794ea0ebd123c7d50fe155c4a7c01be5bafdd10f33623ab53c Jul 23 18:14:42.869471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:42.869434 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hd24d" Jul 23 18:14:43.051489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:43.051435 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hd24d"] Jul 23 18:14:43.054962 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:14:43.054932 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41a242aa_7d7d_42ac_808a_e10fdcf23856.slice/crio-40e767b64bb306733cf47198c296ffd37a1186b2f004360f825fdcceadb1c8cc WatchSource:0}: Error finding container 40e767b64bb306733cf47198c296ffd37a1186b2f004360f825fdcceadb1c8cc: Status 404 returned error can't find the container with id 40e767b64bb306733cf47198c296ffd37a1186b2f004360f825fdcceadb1c8cc Jul 23 18:14:43.571471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:43.571425 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vxtwl" event={"ID":"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b","Type":"ContainerStarted","Data":"e8cdc664d707d5794ea0ebd123c7d50fe155c4a7c01be5bafdd10f33623ab53c"} Jul 23 18:14:43.572878 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:43.572839 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hd24d" event={"ID":"41a242aa-7d7d-42ac-808a-e10fdcf23856","Type":"ContainerStarted","Data":"40e767b64bb306733cf47198c296ffd37a1186b2f004360f825fdcceadb1c8cc"} Jul 23 18:14:45.593490 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:45.593448 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" event={"ID":"b6ca3609-1396-43d0-a027-1e513a686091","Type":"ContainerStarted","Data":"502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504"} Jul 23 18:14:45.594468 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:45.594434 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:45.615412 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:45.615328 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" podStartSLOduration=1.512861054 podStartE2EDuration="4.615307085s" totalImagesPullingTime="3.102446031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:14:45.612232631 +0000 UTC m=+495.309353707" watchObservedRunningTime="2026-07-23 18:14:45.615307085 +0000 UTC m=+495.312428161" Jul 23 18:14:47.605461 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:47.605415 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vxtwl" event={"ID":"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b","Type":"ContainerStarted","Data":"49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7"} Jul 23 18:14:47.606840 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:47.606814 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hd24d" event={"ID":"41a242aa-7d7d-42ac-808a-e10fdcf23856","Type":"ContainerStarted","Data":"fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3"} Jul 23 18:14:47.622598 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:47.622549 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vxtwl" podStartSLOduration=1.266880516 podStartE2EDuration="5.622533555s" totalImagesPullingTime="4.355653039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:14:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:14:47.621199179 +0000 UTC m=+497.318320267" watchObservedRunningTime="2026-07-23 18:14:47.622533555 +0000 UTC m=+497.319654629" Jul 23 18:14:47.635566 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:47.635496 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hd24d" podStartSLOduration=1.515609148 podStartE2EDuration="5.635477067s" totalImagesPullingTime="4.119867919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:14:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:14:47.63518061 +0000 UTC m=+497.332301708" watchObservedRunningTime="2026-07-23 18:14:47.635477067 +0000 UTC m=+497.332598142" Jul 23 18:14:47.660051 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:47.660015 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vxtwl"] Jul 23 18:14:49.617044 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:49.617002 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vxtwl" podUID="7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b" containerName="authorino" containerID="cri-o://49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7" gracePeriod=30 Jul 23 18:14:49.881861 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:49.881777 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vxtwl" Jul 23 18:14:49.935426 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:49.935353 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lr4w5\" (UniqueName: \"kubernetes.io/projected/7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b-kube-api-access-lr4w5\") pod \"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b\" (UID: \"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b\") " Jul 23 18:14:49.937462 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:49.937432 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b-kube-api-access-lr4w5" (OuterVolumeSpecName: "kube-api-access-lr4w5") pod "7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b" (UID: "7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b"). InnerVolumeSpecName "kube-api-access-lr4w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:14:50.036184 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.036140 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr4w5\" (UniqueName: \"kubernetes.io/projected/7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b-kube-api-access-lr4w5\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:50.623774 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.623738 2555 generic.go:356] "Generic (PLEG): container finished" podID="7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b" containerID="49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7" exitCode=0 Jul 23 18:14:50.624185 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.623785 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vxtwl" Jul 23 18:14:50.624185 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.623807 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vxtwl" event={"ID":"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b","Type":"ContainerDied","Data":"49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7"} Jul 23 18:14:50.624185 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.623840 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vxtwl" event={"ID":"7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b","Type":"ContainerDied","Data":"e8cdc664d707d5794ea0ebd123c7d50fe155c4a7c01be5bafdd10f33623ab53c"} Jul 23 18:14:50.624185 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.623856 2555 scope.go:117] "RemoveContainer" containerID="49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7" Jul 23 18:14:50.632848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.632829 2555 scope.go:117] "RemoveContainer" containerID="49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7" Jul 23 18:14:50.633176 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:50.633154 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7\": container with ID starting with 49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7 not found: ID does not exist" containerID="49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7" Jul 23 18:14:50.633259 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.633183 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7"} err="failed to get container status \"49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7\": rpc error: code = NotFound desc = could not find container \"49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7\": container with ID starting with 49f82d1b659e593583eedd4ef4e2f3c62535d1bfb5e9d8e7809e2aa6428889c7 not found: ID does not exist" Jul 23 18:14:50.642694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.642659 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vxtwl"] Jul 23 18:14:50.647540 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:50.647508 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vxtwl"] Jul 23 18:14:52.247884 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:52.247846 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b" path="/var/lib/kubelet/pods/7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b/volumes" Jul 23 18:14:56.600429 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:56.600391 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:57.071325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.071226 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-78t9v"] Jul 23 18:14:57.071545 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.071450 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" podUID="b6ca3609-1396-43d0-a027-1e513a686091" containerName="limitador" containerID="cri-o://502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504" gracePeriod=30 Jul 23 18:14:57.635703 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.635636 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:57.660912 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.660872 2555 generic.go:356] "Generic (PLEG): container finished" podID="b6ca3609-1396-43d0-a027-1e513a686091" containerID="502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504" exitCode=0 Jul 23 18:14:57.661169 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.661020 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" event={"ID":"b6ca3609-1396-43d0-a027-1e513a686091","Type":"ContainerDied","Data":"502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504"} Jul 23 18:14:57.661169 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.661024 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" Jul 23 18:14:57.661169 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.661089 2555 scope.go:117] "RemoveContainer" containerID="502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504" Jul 23 18:14:57.661340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.661074 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-78t9v" event={"ID":"b6ca3609-1396-43d0-a027-1e513a686091","Type":"ContainerDied","Data":"d2ff430063ecf84ed3dfdf7c0fd33a3c41e7740c97c86821353c1a1423accf69"} Jul 23 18:14:57.672360 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.672334 2555 scope.go:117] "RemoveContainer" containerID="502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504" Jul 23 18:14:57.672760 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:14:57.672738 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504\": container with ID starting with 502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504 not found: ID does not exist" containerID="502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504" Jul 23 18:14:57.672816 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.672770 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504"} err="failed to get container status \"502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504\": rpc error: code = NotFound desc = could not find container \"502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504\": container with ID starting with 502cb9f557beed7274031ad3620ca19854a98ab45774974e9565f89131615504 not found: ID does not exist" Jul 23 18:14:57.700225 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.700182 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b6ca3609-1396-43d0-a027-1e513a686091-config-file\") pod \"b6ca3609-1396-43d0-a027-1e513a686091\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " Jul 23 18:14:57.700445 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.700271 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbbmt\" (UniqueName: \"kubernetes.io/projected/b6ca3609-1396-43d0-a027-1e513a686091-kube-api-access-dbbmt\") pod \"b6ca3609-1396-43d0-a027-1e513a686091\" (UID: \"b6ca3609-1396-43d0-a027-1e513a686091\") " Jul 23 18:14:57.700597 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.700566 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6ca3609-1396-43d0-a027-1e513a686091-config-file" (OuterVolumeSpecName: "config-file") pod "b6ca3609-1396-43d0-a027-1e513a686091" (UID: "b6ca3609-1396-43d0-a027-1e513a686091"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:14:57.702277 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.702253 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ca3609-1396-43d0-a027-1e513a686091-kube-api-access-dbbmt" (OuterVolumeSpecName: "kube-api-access-dbbmt") pod "b6ca3609-1396-43d0-a027-1e513a686091" (UID: "b6ca3609-1396-43d0-a027-1e513a686091"). InnerVolumeSpecName "kube-api-access-dbbmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:14:57.801901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.801864 2555 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b6ca3609-1396-43d0-a027-1e513a686091-config-file\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:57.801901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.801896 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbbmt\" (UniqueName: \"kubernetes.io/projected/b6ca3609-1396-43d0-a027-1e513a686091-kube-api-access-dbbmt\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:14:57.985407 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.985354 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-78t9v"] Jul 23 18:14:57.990173 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:57.990144 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-78t9v"] Jul 23 18:14:58.248707 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:14:58.248626 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6ca3609-1396-43d0-a027-1e513a686091" path="/var/lib/kubelet/pods/b6ca3609-1396-43d0-a027-1e513a686091/volumes" Jul 23 18:15:02.698868 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:02.698827 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:02.898892 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:02.898855 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:03.165471 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.165436 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dttrg"] Jul 23 18:15:03.166054 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.166026 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6ca3609-1396-43d0-a027-1e513a686091" containerName="limitador" Jul 23 18:15:03.166054 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.166047 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6ca3609-1396-43d0-a027-1e513a686091" containerName="limitador" Jul 23 18:15:03.166220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.166071 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b" containerName="authorino" Jul 23 18:15:03.166220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.166080 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b" containerName="authorino" Jul 23 18:15:03.166220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.166173 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6ca3609-1396-43d0-a027-1e513a686091" containerName="limitador" Jul 23 18:15:03.166220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.166188 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a0ebbc4-d60d-4e8a-9940-ac1e95932d2b" containerName="authorino" Jul 23 18:15:03.170962 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.170942 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.174377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.174343 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 18:15:03.175442 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.175424 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ljrcc\"" Jul 23 18:15:03.175526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.175465 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 18:15:03.175526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.175427 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 23 18:15:03.179792 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.179768 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dttrg"] Jul 23 18:15:03.208877 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.208842 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:03.250141 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.250103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86j6r\" (UniqueName: \"kubernetes.io/projected/0cfc000c-f667-48a6-9bc9-4c1c6ed961f6-kube-api-access-86j6r\") pod \"postgres-69fb4594cf-dttrg\" (UID: \"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.250305 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.250148 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0cfc000c-f667-48a6-9bc9-4c1c6ed961f6-data\") pod \"postgres-69fb4594cf-dttrg\" (UID: \"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.351199 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.351143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86j6r\" (UniqueName: \"kubernetes.io/projected/0cfc000c-f667-48a6-9bc9-4c1c6ed961f6-kube-api-access-86j6r\") pod \"postgres-69fb4594cf-dttrg\" (UID: \"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.351442 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.351216 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0cfc000c-f667-48a6-9bc9-4c1c6ed961f6-data\") pod \"postgres-69fb4594cf-dttrg\" (UID: \"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.351721 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.351697 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0cfc000c-f667-48a6-9bc9-4c1c6ed961f6-data\") pod \"postgres-69fb4594cf-dttrg\" (UID: \"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.364085 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.364046 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86j6r\" (UniqueName: \"kubernetes.io/projected/0cfc000c-f667-48a6-9bc9-4c1c6ed961f6-kube-api-access-86j6r\") pod \"postgres-69fb4594cf-dttrg\" (UID: \"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.482592 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.482484 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:03.627512 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.627486 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dttrg"] Jul 23 18:15:03.629725 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:15:03.629699 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfc000c_f667_48a6_9bc9_4c1c6ed961f6.slice/crio-658712b2d09bccf4eb17a892fbca326e4f861483e53ac97be0c33bbc5853bdc9 WatchSource:0}: Error finding container 658712b2d09bccf4eb17a892fbca326e4f861483e53ac97be0c33bbc5853bdc9: Status 404 returned error can't find the container with id 658712b2d09bccf4eb17a892fbca326e4f861483e53ac97be0c33bbc5853bdc9 Jul 23 18:15:03.694127 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:03.694091 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" event={"ID":"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6","Type":"ContainerStarted","Data":"658712b2d09bccf4eb17a892fbca326e4f861483e53ac97be0c33bbc5853bdc9"} Jul 23 18:15:04.501316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:04.501284 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:09.734231 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:09.734195 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" event={"ID":"0cfc000c-f667-48a6-9bc9-4c1c6ed961f6","Type":"ContainerStarted","Data":"d2402a64f207e9b8866d374422a7490542bba38d933f54ac798440d25f112f94"} Jul 23 18:15:09.734782 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:09.734297 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:09.754103 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:09.754050 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" podStartSLOduration=1.187056662 podStartE2EDuration="6.754032632s" totalImagesPullingTime="5.56697597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:15:09.752982768 +0000 UTC m=+519.450103868" watchObservedRunningTime="2026-07-23 18:15:09.754032632 +0000 UTC m=+519.451153705" Jul 23 18:15:15.769858 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:15.769828 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dttrg" Jul 23 18:15:16.096161 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.096130 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:16.315829 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.315791 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:16.329022 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.328985 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v84tj"] Jul 23 18:15:16.335436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.335399 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v84tj" Jul 23 18:15:16.337012 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.336982 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v84tj"] Jul 23 18:15:16.471494 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.471396 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtdfl\" (UniqueName: \"kubernetes.io/projected/2eadea66-3105-4549-9ddf-d58afb70f96d-kube-api-access-qtdfl\") pod \"authorino-5d4b67c446-v84tj\" (UID: \"2eadea66-3105-4549-9ddf-d58afb70f96d\") " pod="kuadrant-system/authorino-5d4b67c446-v84tj" Jul 23 18:15:16.564742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.564702 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v84tj"] Jul 23 18:15:16.564993 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:16.564975 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qtdfl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-v84tj" podUID="2eadea66-3105-4549-9ddf-d58afb70f96d" Jul 23 18:15:16.572117 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.572079 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtdfl\" (UniqueName: \"kubernetes.io/projected/2eadea66-3105-4549-9ddf-d58afb70f96d-kube-api-access-qtdfl\") pod \"authorino-5d4b67c446-v84tj\" (UID: \"2eadea66-3105-4549-9ddf-d58afb70f96d\") " pod="kuadrant-system/authorino-5d4b67c446-v84tj" Jul 23 18:15:16.585525 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.585497 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtdfl\" (UniqueName: \"kubernetes.io/projected/2eadea66-3105-4549-9ddf-d58afb70f96d-kube-api-access-qtdfl\") pod \"authorino-5d4b67c446-v84tj\" (UID: \"2eadea66-3105-4549-9ddf-d58afb70f96d\") " pod="kuadrant-system/authorino-5d4b67c446-v84tj" Jul 23 18:15:16.595085 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.595051 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76879c68f7-hxxxz"] Jul 23 18:15:16.598507 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.598487 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76879c68f7-hxxxz" Jul 23 18:15:16.604658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.604627 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76879c68f7-hxxxz"] Jul 23 18:15:16.672737 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.672698 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck4cd\" (UniqueName: \"kubernetes.io/projected/3cec99e6-803f-41e8-8c3c-40a3312bae33-kube-api-access-ck4cd\") pod \"authorino-76879c68f7-hxxxz\" (UID: \"3cec99e6-803f-41e8-8c3c-40a3312bae33\") " pod="kuadrant-system/authorino-76879c68f7-hxxxz" Jul 23 18:15:16.708696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.708659 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76879c68f7-hxxxz"] Jul 23 18:15:16.708944 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:16.708923 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ck4cd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76879c68f7-hxxxz" podUID="3cec99e6-803f-41e8-8c3c-40a3312bae33" Jul 23 18:15:16.769059 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.768966 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76879c68f7-hxxxz" Jul 23 18:15:16.769234 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.768971 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v84tj" Jul 23 18:15:16.769932 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.769907 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-566759f877-9dxzm"] Jul 23 18:15:16.773359 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.773311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck4cd\" (UniqueName: \"kubernetes.io/projected/3cec99e6-803f-41e8-8c3c-40a3312bae33-kube-api-access-ck4cd\") pod \"authorino-76879c68f7-hxxxz\" (UID: \"3cec99e6-803f-41e8-8c3c-40a3312bae33\") " pod="kuadrant-system/authorino-76879c68f7-hxxxz" Jul 23 18:15:16.773572 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.773552 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:16.775916 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.775897 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76879c68f7-hxxxz" Jul 23 18:15:16.776266 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.776252 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 18:15:16.779694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.779676 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v84tj" Jul 23 18:15:16.787904 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.787878 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-566759f877-9dxzm"] Jul 23 18:15:16.795539 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.795516 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck4cd\" (UniqueName: \"kubernetes.io/projected/3cec99e6-803f-41e8-8c3c-40a3312bae33-kube-api-access-ck4cd\") pod \"authorino-76879c68f7-hxxxz\" (UID: \"3cec99e6-803f-41e8-8c3c-40a3312bae33\") " pod="kuadrant-system/authorino-76879c68f7-hxxxz" Jul 23 18:15:16.874500 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.874461 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck4cd\" (UniqueName: \"kubernetes.io/projected/3cec99e6-803f-41e8-8c3c-40a3312bae33-kube-api-access-ck4cd\") pod \"3cec99e6-803f-41e8-8c3c-40a3312bae33\" (UID: \"3cec99e6-803f-41e8-8c3c-40a3312bae33\") " Jul 23 18:15:16.874696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.874560 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtdfl\" (UniqueName: \"kubernetes.io/projected/2eadea66-3105-4549-9ddf-d58afb70f96d-kube-api-access-qtdfl\") pod \"2eadea66-3105-4549-9ddf-d58afb70f96d\" (UID: \"2eadea66-3105-4549-9ddf-d58afb70f96d\") " Jul 23 18:15:16.874696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.874664 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvnrp\" (UniqueName: \"kubernetes.io/projected/2a7b506d-3af4-4790-978f-6a3d8b717e30-kube-api-access-nvnrp\") pod \"authorino-566759f877-9dxzm\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:16.874790 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.874731 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a7b506d-3af4-4790-978f-6a3d8b717e30-tls-cert\") pod \"authorino-566759f877-9dxzm\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:16.876744 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.876713 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eadea66-3105-4549-9ddf-d58afb70f96d-kube-api-access-qtdfl" (OuterVolumeSpecName: "kube-api-access-qtdfl") pod "2eadea66-3105-4549-9ddf-d58afb70f96d" (UID: "2eadea66-3105-4549-9ddf-d58afb70f96d"). InnerVolumeSpecName "kube-api-access-qtdfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:15:16.876862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.876751 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cec99e6-803f-41e8-8c3c-40a3312bae33-kube-api-access-ck4cd" (OuterVolumeSpecName: "kube-api-access-ck4cd") pod "3cec99e6-803f-41e8-8c3c-40a3312bae33" (UID: "3cec99e6-803f-41e8-8c3c-40a3312bae33"). InnerVolumeSpecName "kube-api-access-ck4cd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:15:16.975777 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.975736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a7b506d-3af4-4790-978f-6a3d8b717e30-tls-cert\") pod \"authorino-566759f877-9dxzm\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:16.975932 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.975800 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvnrp\" (UniqueName: \"kubernetes.io/projected/2a7b506d-3af4-4790-978f-6a3d8b717e30-kube-api-access-nvnrp\") pod \"authorino-566759f877-9dxzm\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:16.975932 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.975841 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtdfl\" (UniqueName: \"kubernetes.io/projected/2eadea66-3105-4549-9ddf-d58afb70f96d-kube-api-access-qtdfl\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:15:16.975932 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.975853 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck4cd\" (UniqueName: \"kubernetes.io/projected/3cec99e6-803f-41e8-8c3c-40a3312bae33-kube-api-access-ck4cd\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:15:16.978250 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.978230 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a7b506d-3af4-4790-978f-6a3d8b717e30-tls-cert\") pod \"authorino-566759f877-9dxzm\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:16.986407 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:16.986354 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvnrp\" (UniqueName: \"kubernetes.io/projected/2a7b506d-3af4-4790-978f-6a3d8b717e30-kube-api-access-nvnrp\") pod \"authorino-566759f877-9dxzm\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:17.085235 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.085147 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:15:17.224762 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.224713 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-566759f877-9dxzm"] Jul 23 18:15:17.227225 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:15:17.227158 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a7b506d_3af4_4790_978f_6a3d8b717e30.slice/crio-a69af024a66ff647bdf6e97c23a091e87997248eb2d8da7d9d217b1dc037eb83 WatchSource:0}: Error finding container a69af024a66ff647bdf6e97c23a091e87997248eb2d8da7d9d217b1dc037eb83: Status 404 returned error can't find the container with id a69af024a66ff647bdf6e97c23a091e87997248eb2d8da7d9d217b1dc037eb83 Jul 23 18:15:17.778465 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.778021 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566759f877-9dxzm" event={"ID":"2a7b506d-3af4-4790-978f-6a3d8b717e30","Type":"ContainerStarted","Data":"9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f"} Jul 23 18:15:17.778465 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.778080 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566759f877-9dxzm" event={"ID":"2a7b506d-3af4-4790-978f-6a3d8b717e30","Type":"ContainerStarted","Data":"a69af024a66ff647bdf6e97c23a091e87997248eb2d8da7d9d217b1dc037eb83"} Jul 23 18:15:17.778465 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.778193 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76879c68f7-hxxxz" Jul 23 18:15:17.778465 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.778327 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-v84tj" Jul 23 18:15:17.796093 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.796043 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-566759f877-9dxzm" podStartSLOduration=1.414429557 podStartE2EDuration="1.79602454s" totalImagesPullingTime="381.594983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:15:17.79482547 +0000 UTC m=+527.491946564" watchObservedRunningTime="2026-07-23 18:15:17.79602454 +0000 UTC m=+527.493145613" Jul 23 18:15:17.822649 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.822616 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76879c68f7-hxxxz"] Jul 23 18:15:17.835282 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.835247 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hd24d"] Jul 23 18:15:17.835513 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.835488 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hd24d" podUID="41a242aa-7d7d-42ac-808a-e10fdcf23856" containerName="authorino" containerID="cri-o://fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3" gracePeriod=30 Jul 23 18:15:17.837906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.837877 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76879c68f7-hxxxz"] Jul 23 18:15:17.865340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.865258 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v84tj"] Jul 23 18:15:17.868543 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:17.868515 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-v84tj"] Jul 23 18:15:18.077999 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.077970 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hd24d" Jul 23 18:15:18.186035 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.185947 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5msv6\" (UniqueName: \"kubernetes.io/projected/41a242aa-7d7d-42ac-808a-e10fdcf23856-kube-api-access-5msv6\") pod \"41a242aa-7d7d-42ac-808a-e10fdcf23856\" (UID: \"41a242aa-7d7d-42ac-808a-e10fdcf23856\") " Jul 23 18:15:18.188037 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.188004 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a242aa-7d7d-42ac-808a-e10fdcf23856-kube-api-access-5msv6" (OuterVolumeSpecName: "kube-api-access-5msv6") pod "41a242aa-7d7d-42ac-808a-e10fdcf23856" (UID: "41a242aa-7d7d-42ac-808a-e10fdcf23856"). InnerVolumeSpecName "kube-api-access-5msv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:15:18.255116 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.255080 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eadea66-3105-4549-9ddf-d58afb70f96d" path="/var/lib/kubelet/pods/2eadea66-3105-4549-9ddf-d58afb70f96d/volumes" Jul 23 18:15:18.255310 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.255298 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cec99e6-803f-41e8-8c3c-40a3312bae33" path="/var/lib/kubelet/pods/3cec99e6-803f-41e8-8c3c-40a3312bae33/volumes" Jul 23 18:15:18.287385 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.287342 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5msv6\" (UniqueName: \"kubernetes.io/projected/41a242aa-7d7d-42ac-808a-e10fdcf23856-kube-api-access-5msv6\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:15:18.786261 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.786222 2555 generic.go:356] "Generic (PLEG): container finished" podID="41a242aa-7d7d-42ac-808a-e10fdcf23856" containerID="fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3" exitCode=0 Jul 23 18:15:18.786763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.786283 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hd24d" Jul 23 18:15:18.786763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.786302 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hd24d" event={"ID":"41a242aa-7d7d-42ac-808a-e10fdcf23856","Type":"ContainerDied","Data":"fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3"} Jul 23 18:15:18.786763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.786346 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hd24d" event={"ID":"41a242aa-7d7d-42ac-808a-e10fdcf23856","Type":"ContainerDied","Data":"40e767b64bb306733cf47198c296ffd37a1186b2f004360f825fdcceadb1c8cc"} Jul 23 18:15:18.786763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.786377 2555 scope.go:117] "RemoveContainer" containerID="fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3" Jul 23 18:15:18.795182 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.795159 2555 scope.go:117] "RemoveContainer" containerID="fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3" Jul 23 18:15:18.795571 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:18.795547 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3\": container with ID starting with fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3 not found: ID does not exist" containerID="fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3" Jul 23 18:15:18.795669 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.795577 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3"} err="failed to get container status \"fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3\": rpc error: code = NotFound desc = could not find container \"fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3\": container with ID starting with fc2bc70a8259ecea47f9f30ae7fd54217b2c914a6144ce45e4185024cf762ec3 not found: ID does not exist" Jul 23 18:15:18.804324 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.804297 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hd24d"] Jul 23 18:15:18.809881 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:18.809854 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hd24d"] Jul 23 18:15:20.247665 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:20.247630 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a242aa-7d7d-42ac-808a-e10fdcf23856" path="/var/lib/kubelet/pods/41a242aa-7d7d-42ac-808a-e10fdcf23856/volumes" Jul 23 18:15:22.544619 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.544531 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:22.554282 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.554245 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b4f99c47-jqljw"] Jul 23 18:15:22.554679 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.554667 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a242aa-7d7d-42ac-808a-e10fdcf23856" containerName="authorino" Jul 23 18:15:22.554728 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.554682 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a242aa-7d7d-42ac-808a-e10fdcf23856" containerName="authorino" Jul 23 18:15:22.554763 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.554747 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a242aa-7d7d-42ac-808a-e10fdcf23856" containerName="authorino" Jul 23 18:15:22.559126 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.559107 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:22.561880 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.561853 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 18:15:22.561880 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.561878 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 18:15:22.562077 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.561924 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pfz6r\"" Jul 23 18:15:22.569606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.569581 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b4f99c47-jqljw"] Jul 23 18:15:22.663649 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.663611 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:22.730383 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.730338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbm6\" (UniqueName: \"kubernetes.io/projected/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-kube-api-access-vmbm6\") pod \"maas-controller-67b4f99c47-jqljw\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:22.730596 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.730402 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert\") pod \"maas-controller-67b4f99c47-jqljw\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:22.740046 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.740014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:22.752854 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.752823 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f469bfccf-4d72b"] Jul 23 18:15:22.756225 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.756207 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:22.773568 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.773534 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f469bfccf-4d72b"] Jul 23 18:15:22.831853 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.831742 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbm6\" (UniqueName: \"kubernetes.io/projected/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-kube-api-access-vmbm6\") pod \"maas-controller-67b4f99c47-jqljw\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:22.831853 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.831803 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert\") pod \"maas-controller-67b4f99c47-jqljw\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:22.832057 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:22.831927 2555 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 23 18:15:22.832057 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:22.832016 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert podName:46e12ffd-56d4-4f19-8d65-dec41cd25dc2 nodeName:}" failed. No retries permitted until 2026-07-23 18:15:23.33199502 +0000 UTC m=+533.029116075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert") pod "maas-controller-67b4f99c47-jqljw" (UID: "46e12ffd-56d4-4f19-8d65-dec41cd25dc2") : secret "maas-controller-webhook-cert" not found Jul 23 18:15:22.845637 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.845602 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbm6\" (UniqueName: \"kubernetes.io/projected/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-kube-api-access-vmbm6\") pod \"maas-controller-67b4f99c47-jqljw\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:22.932878 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.932838 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9hp\" (UniqueName: \"kubernetes.io/projected/b436aad2-51df-4389-9058-d6568746b604-kube-api-access-9h9hp\") pod \"maas-controller-7f469bfccf-4d72b\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:22.933056 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:22.932983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert\") pod \"maas-controller-7f469bfccf-4d72b\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:23.033469 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.033429 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert\") pod \"maas-controller-7f469bfccf-4d72b\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:23.033671 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.033520 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9hp\" (UniqueName: \"kubernetes.io/projected/b436aad2-51df-4389-9058-d6568746b604-kube-api-access-9h9hp\") pod \"maas-controller-7f469bfccf-4d72b\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:23.033671 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:23.033587 2555 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 23 18:15:23.033671 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:23.033655 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert podName:b436aad2-51df-4389-9058-d6568746b604 nodeName:}" failed. No retries permitted until 2026-07-23 18:15:23.533637782 +0000 UTC m=+533.230758847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert") pod "maas-controller-7f469bfccf-4d72b" (UID: "b436aad2-51df-4389-9058-d6568746b604") : secret "maas-controller-webhook-cert" not found Jul 23 18:15:23.044233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.044203 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9hp\" (UniqueName: \"kubernetes.io/projected/b436aad2-51df-4389-9058-d6568746b604-kube-api-access-9h9hp\") pod \"maas-controller-7f469bfccf-4d72b\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:23.180616 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.180578 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:23.336194 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.336153 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert\") pod \"maas-controller-67b4f99c47-jqljw\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:23.338634 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.338596 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert\") pod \"maas-controller-67b4f99c47-jqljw\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:23.471667 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.471561 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:23.538748 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.538691 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert\") pod \"maas-controller-7f469bfccf-4d72b\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:23.542020 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.541992 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert\") pod \"maas-controller-7f469bfccf-4d72b\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:23.620908 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.620880 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b4f99c47-jqljw"] Jul 23 18:15:23.622956 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:15:23.622929 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e12ffd_56d4_4f19_8d65_dec41cd25dc2.slice/crio-da54c279950a9f014638f65c90b58e4d313465ca6cc80a94650fb9d3932f29f1 WatchSource:0}: Error finding container da54c279950a9f014638f65c90b58e4d313465ca6cc80a94650fb9d3932f29f1: Status 404 returned error can't find the container with id da54c279950a9f014638f65c90b58e4d313465ca6cc80a94650fb9d3932f29f1 Jul 23 18:15:23.666674 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.666635 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:23.820930 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.820894 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b4f99c47-jqljw" event={"ID":"46e12ffd-56d4-4f19-8d65-dec41cd25dc2","Type":"ContainerStarted","Data":"da54c279950a9f014638f65c90b58e4d313465ca6cc80a94650fb9d3932f29f1"} Jul 23 18:15:23.832341 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:23.832317 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f469bfccf-4d72b"] Jul 23 18:15:23.834639 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:15:23.834601 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb436aad2_51df_4389_9058_d6568746b604.slice/crio-56d1bcd918ff78921b372fd11bc2fb5b00b1005a1f16c4406528de445fc3a508 WatchSource:0}: Error finding container 56d1bcd918ff78921b372fd11bc2fb5b00b1005a1f16c4406528de445fc3a508: Status 404 returned error can't find the container with id 56d1bcd918ff78921b372fd11bc2fb5b00b1005a1f16c4406528de445fc3a508 Jul 23 18:15:24.837846 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:24.837741 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f469bfccf-4d72b" event={"ID":"b436aad2-51df-4389-9058-d6568746b604","Type":"ContainerStarted","Data":"56d1bcd918ff78921b372fd11bc2fb5b00b1005a1f16c4406528de445fc3a508"} Jul 23 18:15:26.389490 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.389427 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:26.498175 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.498037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:26.850921 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.850884 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f469bfccf-4d72b" event={"ID":"b436aad2-51df-4389-9058-d6568746b604","Type":"ContainerStarted","Data":"a5113c48563338a1830e2a523596d453cdbcc77da8ebca8292d063a56a3b124e"} Jul 23 18:15:26.851121 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.851100 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:26.853027 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.852974 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b4f99c47-jqljw" event={"ID":"46e12ffd-56d4-4f19-8d65-dec41cd25dc2","Type":"ContainerStarted","Data":"ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd"} Jul 23 18:15:26.853154 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.853058 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:26.879734 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.879681 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f469bfccf-4d72b" podStartSLOduration=2.532910003 podStartE2EDuration="4.879663651s" totalImagesPullingTime="2.346753648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:15:26.877151527 +0000 UTC m=+536.574272624" watchObservedRunningTime="2026-07-23 18:15:26.879663651 +0000 UTC m=+536.576784740" Jul 23 18:15:26.910099 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:26.910041 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b4f99c47-jqljw" podStartSLOduration=2.358417961 podStartE2EDuration="4.910023553s" totalImagesPullingTime="2.551605592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:15:26.907752755 +0000 UTC m=+536.604873839" watchObservedRunningTime="2026-07-23 18:15:26.910023553 +0000 UTC m=+536.607144627" Jul 23 18:15:37.864846 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:37.864813 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:15:37.865409 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:37.864902 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:37.938691 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:37.938658 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b4f99c47-jqljw"] Jul 23 18:15:37.938946 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:37.938912 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b4f99c47-jqljw" podUID="46e12ffd-56d4-4f19-8d65-dec41cd25dc2" containerName="manager" containerID="cri-o://ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd" gracePeriod=30 Jul 23 18:15:38.203530 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.203499 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:38.356248 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.356212 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert\") pod \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " Jul 23 18:15:38.356484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.356277 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmbm6\" (UniqueName: \"kubernetes.io/projected/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-kube-api-access-vmbm6\") pod \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\" (UID: \"46e12ffd-56d4-4f19-8d65-dec41cd25dc2\") " Jul 23 18:15:38.358546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.358516 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-kube-api-access-vmbm6" (OuterVolumeSpecName: "kube-api-access-vmbm6") pod "46e12ffd-56d4-4f19-8d65-dec41cd25dc2" (UID: "46e12ffd-56d4-4f19-8d65-dec41cd25dc2"). InnerVolumeSpecName "kube-api-access-vmbm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:15:38.358651 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.358523 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "46e12ffd-56d4-4f19-8d65-dec41cd25dc2" (UID: "46e12ffd-56d4-4f19-8d65-dec41cd25dc2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:15:38.457150 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.457066 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-webhook-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:15:38.457150 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.457099 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmbm6\" (UniqueName: \"kubernetes.io/projected/46e12ffd-56d4-4f19-8d65-dec41cd25dc2-kube-api-access-vmbm6\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:15:38.918535 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.918498 2555 generic.go:356] "Generic (PLEG): container finished" podID="46e12ffd-56d4-4f19-8d65-dec41cd25dc2" containerID="ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd" exitCode=0 Jul 23 18:15:38.918987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.918544 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b4f99c47-jqljw" Jul 23 18:15:38.918987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.918576 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b4f99c47-jqljw" event={"ID":"46e12ffd-56d4-4f19-8d65-dec41cd25dc2","Type":"ContainerDied","Data":"ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd"} Jul 23 18:15:38.918987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.918624 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b4f99c47-jqljw" event={"ID":"46e12ffd-56d4-4f19-8d65-dec41cd25dc2","Type":"ContainerDied","Data":"da54c279950a9f014638f65c90b58e4d313465ca6cc80a94650fb9d3932f29f1"} Jul 23 18:15:38.918987 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.918646 2555 scope.go:117] "RemoveContainer" containerID="ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd" Jul 23 18:15:38.929604 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.928867 2555 scope.go:117] "RemoveContainer" containerID="ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd" Jul 23 18:15:38.929604 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:15:38.929305 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd\": container with ID starting with ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd not found: ID does not exist" containerID="ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd" Jul 23 18:15:38.929927 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.929346 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd"} err="failed to get container status \"ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd\": rpc error: code = NotFound desc = could not find container \"ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd\": container with ID starting with ea6b8601bef8c62241bf88c16dafc5146022663c48468bf6a56e9bf141e510fd not found: ID does not exist" Jul 23 18:15:38.946708 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.946676 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b4f99c47-jqljw"] Jul 23 18:15:38.952456 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:38.952414 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b4f99c47-jqljw"] Jul 23 18:15:40.247724 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:40.247690 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46e12ffd-56d4-4f19-8d65-dec41cd25dc2" path="/var/lib/kubelet/pods/46e12ffd-56d4-4f19-8d65-dec41cd25dc2/volumes" Jul 23 18:15:44.543719 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:44.543686 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:44.601148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:44.601103 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:57.938091 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:57.938052 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:58.038241 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:58.038207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:58.642566 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:58.642530 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:58.907168 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:58.907083 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:59.343813 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.343782 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:59.439537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.439509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:15:59.523486 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.523452 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-vk65d"] Jul 23 18:15:59.523894 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.523879 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e12ffd-56d4-4f19-8d65-dec41cd25dc2" containerName="manager" Jul 23 18:15:59.523894 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.523894 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e12ffd-56d4-4f19-8d65-dec41cd25dc2" containerName="manager" Jul 23 18:15:59.524004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.523969 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="46e12ffd-56d4-4f19-8d65-dec41cd25dc2" containerName="manager" Jul 23 18:15:59.528454 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.528434 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.531403 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.531382 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qrcss\"" Jul 23 18:15:59.531744 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.531728 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 18:15:59.538189 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.538165 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-vk65d"] Jul 23 18:15:59.643224 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.643148 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7btg\" (UniqueName: \"kubernetes.io/projected/529670ef-6888-4e7c-8570-d33ba74800d0-kube-api-access-v7btg\") pod \"payload-pre-processing-7677b774bb-vk65d\" (UID: \"529670ef-6888-4e7c-8570-d33ba74800d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.643224 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.643213 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/529670ef-6888-4e7c-8570-d33ba74800d0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-vk65d\" (UID: \"529670ef-6888-4e7c-8570-d33ba74800d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.744126 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.744084 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/529670ef-6888-4e7c-8570-d33ba74800d0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-vk65d\" (UID: \"529670ef-6888-4e7c-8570-d33ba74800d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.744306 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.744175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7btg\" (UniqueName: \"kubernetes.io/projected/529670ef-6888-4e7c-8570-d33ba74800d0-kube-api-access-v7btg\") pod \"payload-pre-processing-7677b774bb-vk65d\" (UID: \"529670ef-6888-4e7c-8570-d33ba74800d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.744820 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.744799 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/529670ef-6888-4e7c-8570-d33ba74800d0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-vk65d\" (UID: \"529670ef-6888-4e7c-8570-d33ba74800d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.756384 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.756335 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7btg\" (UniqueName: \"kubernetes.io/projected/529670ef-6888-4e7c-8570-d33ba74800d0-kube-api-access-v7btg\") pod \"payload-pre-processing-7677b774bb-vk65d\" (UID: \"529670ef-6888-4e7c-8570-d33ba74800d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.841049 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.841016 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:15:59.981297 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:15:59.981270 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-vk65d"] Jul 23 18:15:59.983292 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:15:59.983263 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod529670ef_6888_4e7c_8570_d33ba74800d0.slice/crio-78bb63f54f73a152dc6d1dc7762224161a9a8944135bb9c0b364ea6588b08614 WatchSource:0}: Error finding container 78bb63f54f73a152dc6d1dc7762224161a9a8944135bb9c0b364ea6588b08614: Status 404 returned error can't find the container with id 78bb63f54f73a152dc6d1dc7762224161a9a8944135bb9c0b364ea6588b08614 Jul 23 18:16:00.026977 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:00.026940 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" event={"ID":"529670ef-6888-4e7c-8570-d33ba74800d0","Type":"ContainerStarted","Data":"78bb63f54f73a152dc6d1dc7762224161a9a8944135bb9c0b364ea6588b08614"} Jul 23 18:16:02.040135 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:02.040047 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" event={"ID":"529670ef-6888-4e7c-8570-d33ba74800d0","Type":"ContainerStarted","Data":"f82bcfc2b15f017ecaee1181776bf3d07b55d562643d124522fbf1f49e5b45a9"} Jul 23 18:16:02.040135 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:02.040114 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:16:02.061160 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:02.061115 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" podStartSLOduration=1.382488263 podStartE2EDuration="3.061099968s" totalImagesPullingTime="1.678611705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:16:02.060077222 +0000 UTC m=+571.757198295" watchObservedRunningTime="2026-07-23 18:16:02.061099968 +0000 UTC m=+571.758221041" Jul 23 18:16:10.691319 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:10.691234 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:10.786516 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:10.786484 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:12.684044 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:12.683997 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:13.047004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:13.046928 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-vk65d" Jul 23 18:16:24.721798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.721767 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll"] Jul 23 18:16:24.725018 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.724999 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.729049 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.729025 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 18:16:24.729174 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.729148 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 23 18:16:24.729248 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.729032 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 18:16:24.729301 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.729110 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dj6jj\"" Jul 23 18:16:24.738255 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.738222 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll"] Jul 23 18:16:24.769494 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.769466 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.769668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.769501 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.769668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.769593 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.769668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.769647 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.769806 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.769673 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-524lz\" (UniqueName: \"kubernetes.io/projected/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-kube-api-access-524lz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.769806 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.769701 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.782074 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.782050 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:24.870779 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.870747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.870942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.870785 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.870942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.870810 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-524lz\" (UniqueName: \"kubernetes.io/projected/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-kube-api-access-524lz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.870942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.870850 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.870942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.870931 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.871110 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.870953 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.871273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.871221 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.871347 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.871278 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.871423 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.871344 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.873112 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.873089 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.873358 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.873339 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:24.880759 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:24.880735 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-524lz\" (UniqueName: \"kubernetes.io/projected/db2ee93e-bdab-4e5d-bffd-0e36d042fd25-kube-api-access-524lz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll\" (UID: \"db2ee93e-bdab-4e5d-bffd-0e36d042fd25\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:25.036874 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:25.036784 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:25.189596 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:25.189570 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll"] Jul 23 18:16:25.191165 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:16:25.191128 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2ee93e_bdab_4e5d_bffd_0e36d042fd25.slice/crio-281b992461cabb3a2d49105bf162ee393d9f3068795a54df43904e8abb35591f WatchSource:0}: Error finding container 281b992461cabb3a2d49105bf162ee393d9f3068795a54df43904e8abb35591f: Status 404 returned error can't find the container with id 281b992461cabb3a2d49105bf162ee393d9f3068795a54df43904e8abb35591f Jul 23 18:16:26.165835 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:26.165801 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" event={"ID":"db2ee93e-bdab-4e5d-bffd-0e36d042fd25","Type":"ContainerStarted","Data":"281b992461cabb3a2d49105bf162ee393d9f3068795a54df43904e8abb35591f"} Jul 23 18:16:30.190042 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:30.190004 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" event={"ID":"db2ee93e-bdab-4e5d-bffd-0e36d042fd25","Type":"ContainerStarted","Data":"c964111a7ed2df805cb564b44664925e649bd1a8ba6ef49f7da869fc89b84bfd"} Jul 23 18:16:30.212361 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:30.212305 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" podStartSLOduration=2.17743642 podStartE2EDuration="6.212290398s" totalImagesPullingTime="4.034853978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:16:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:16:30.209763857 +0000 UTC m=+599.906884951" watchObservedRunningTime="2026-07-23 18:16:30.212290398 +0000 UTC m=+599.909411478" Jul 23 18:16:31.962762 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:31.962730 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:16:31.963282 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:31.963260 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:16:35.037836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:35.037798 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:35.038261 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:35.037873 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:35.050618 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:35.050587 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:35.227170 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:35.227137 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll" Jul 23 18:16:36.385174 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.385136 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n"] Jul 23 18:16:36.388738 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.388721 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.391281 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.391265 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 23 18:16:36.400223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.400201 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n"] Jul 23 18:16:36.480583 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.480539 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.480583 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.480588 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.480772 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.480615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.480772 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.480720 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.480772 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.480753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/757a3d73-ed8a-4285-9604-f4d595cc4ab3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.480891 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.480833 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv99z\" (UniqueName: \"kubernetes.io/projected/757a3d73-ed8a-4285-9604-f4d595cc4ab3-kube-api-access-hv99z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.539908 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.539872 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:36.582235 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582264 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582291 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/757a3d73-ed8a-4285-9604-f4d595cc4ab3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582355 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv99z\" (UniqueName: \"kubernetes.io/projected/757a3d73-ed8a-4285-9604-f4d595cc4ab3-kube-api-access-hv99z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582405 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582440 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582435 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582813 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582780 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582783 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.582893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.582850 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.584484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.584462 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/757a3d73-ed8a-4285-9604-f4d595cc4ab3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.585006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.584987 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/757a3d73-ed8a-4285-9604-f4d595cc4ab3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.593231 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.593211 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv99z\" (UniqueName: \"kubernetes.io/projected/757a3d73-ed8a-4285-9604-f4d595cc4ab3-kube-api-access-hv99z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q2x2n\" (UID: \"757a3d73-ed8a-4285-9604-f4d595cc4ab3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.698511 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.698429 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:36.837996 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:36.837968 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n"] Jul 23 18:16:36.841057 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:16:36.841028 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757a3d73_ed8a_4285_9604_f4d595cc4ab3.slice/crio-31cfcf0fff6a163620f7791f16a9f876ad43e1b70612623f4082ade0dae32f48 WatchSource:0}: Error finding container 31cfcf0fff6a163620f7791f16a9f876ad43e1b70612623f4082ade0dae32f48: Status 404 returned error can't find the container with id 31cfcf0fff6a163620f7791f16a9f876ad43e1b70612623f4082ade0dae32f48 Jul 23 18:16:37.230894 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:37.230857 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" event={"ID":"757a3d73-ed8a-4285-9604-f4d595cc4ab3","Type":"ContainerStarted","Data":"f05d4eebd77d07a878e234a868c2ec95f64fb21226baaf11aa6c746111362d57"} Jul 23 18:16:37.231057 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:37.230921 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" event={"ID":"757a3d73-ed8a-4285-9604-f4d595cc4ab3","Type":"ContainerStarted","Data":"31cfcf0fff6a163620f7791f16a9f876ad43e1b70612623f4082ade0dae32f48"} Jul 23 18:16:37.250662 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:37.250610 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" podStartSLOduration=1.006189444 podStartE2EDuration="1.250594659s" totalImagesPullingTime="244.405215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:16:37.249710517 +0000 UTC m=+606.946831596" watchObservedRunningTime="2026-07-23 18:16:37.250594659 +0000 UTC m=+606.947715732" Jul 23 18:16:46.288968 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:46.288926 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:16:46.699012 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:46.698968 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:46.699214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:46.699096 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:46.711703 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:46.711670 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:47.296915 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:47.296878 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q2x2n" Jul 23 18:16:53.171180 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:16:53.171102 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:17:13.364853 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:17:13.364820 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:17:13.928094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:17:13.928061 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:17:21.913121 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:17:21.913087 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:17:31.584839 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:17:31.584799 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:19.904242 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:19.904207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:41.254241 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.254196 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bf6c8c6d-cbpfx"] Jul 23 18:18:41.258086 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.258061 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.266163 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.266132 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bf6c8c6d-cbpfx"] Jul 23 18:18:41.340351 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.340315 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed9cccda-fdca-454a-ba9c-c1027f75b1d1-tls-cert\") pod \"authorino-7bf6c8c6d-cbpfx\" (UID: \"ed9cccda-fdca-454a-ba9c-c1027f75b1d1\") " pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.340559 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.340421 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7cz9\" (UniqueName: \"kubernetes.io/projected/ed9cccda-fdca-454a-ba9c-c1027f75b1d1-kube-api-access-r7cz9\") pod \"authorino-7bf6c8c6d-cbpfx\" (UID: \"ed9cccda-fdca-454a-ba9c-c1027f75b1d1\") " pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.441651 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.441610 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed9cccda-fdca-454a-ba9c-c1027f75b1d1-tls-cert\") pod \"authorino-7bf6c8c6d-cbpfx\" (UID: \"ed9cccda-fdca-454a-ba9c-c1027f75b1d1\") " pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.441842 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.441691 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7cz9\" (UniqueName: \"kubernetes.io/projected/ed9cccda-fdca-454a-ba9c-c1027f75b1d1-kube-api-access-r7cz9\") pod \"authorino-7bf6c8c6d-cbpfx\" (UID: \"ed9cccda-fdca-454a-ba9c-c1027f75b1d1\") " pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.444251 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.444213 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed9cccda-fdca-454a-ba9c-c1027f75b1d1-tls-cert\") pod \"authorino-7bf6c8c6d-cbpfx\" (UID: \"ed9cccda-fdca-454a-ba9c-c1027f75b1d1\") " pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.450905 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.450864 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7cz9\" (UniqueName: \"kubernetes.io/projected/ed9cccda-fdca-454a-ba9c-c1027f75b1d1-kube-api-access-r7cz9\") pod \"authorino-7bf6c8c6d-cbpfx\" (UID: \"ed9cccda-fdca-454a-ba9c-c1027f75b1d1\") " pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.568974 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.568889 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" Jul 23 18:18:41.633279 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.633248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:41.719330 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.719297 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bf6c8c6d-cbpfx"] Jul 23 18:18:41.721266 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:18:41.721224 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded9cccda_fdca_454a_ba9c_c1027f75b1d1.slice/crio-bfe91cace3703a79350edf76486ba0e37d3946c142e750afbb618a8b5d9ca28d WatchSource:0}: Error finding container bfe91cace3703a79350edf76486ba0e37d3946c142e750afbb618a8b5d9ca28d: Status 404 returned error can't find the container with id bfe91cace3703a79350edf76486ba0e37d3946c142e750afbb618a8b5d9ca28d Jul 23 18:18:41.721751 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.721726 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:41.722922 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.722903 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:18:41.891424 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:41.891383 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" event={"ID":"ed9cccda-fdca-454a-ba9c-c1027f75b1d1","Type":"ContainerStarted","Data":"bfe91cace3703a79350edf76486ba0e37d3946c142e750afbb618a8b5d9ca28d"} Jul 23 18:18:42.765063 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:42.765026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:42.865848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:42.865813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:18:42.899560 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:42.899522 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" event={"ID":"ed9cccda-fdca-454a-ba9c-c1027f75b1d1","Type":"ContainerStarted","Data":"ebdd408b33a48895b839e5514a50c8f8a5d04a39b93be1ff6c517258e0a96f0d"} Jul 23 18:18:42.923391 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:42.923250 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bf6c8c6d-cbpfx" podStartSLOduration=0.993715273 podStartE2EDuration="1.923225983s" totalImagesPullingTime="929.51071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:18:42.920889979 +0000 UTC m=+732.618011090" watchObservedRunningTime="2026-07-23 18:18:42.923225983 +0000 UTC m=+732.620347058" Jul 23 18:18:42.960940 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:42.960897 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-566759f877-9dxzm"] Jul 23 18:18:42.961191 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:42.961160 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-566759f877-9dxzm" podUID="2a7b506d-3af4-4790-978f-6a3d8b717e30" containerName="authorino" containerID="cri-o://9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f" gracePeriod=30 Jul 23 18:18:43.231619 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.231593 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:18:43.257817 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.256771 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a7b506d-3af4-4790-978f-6a3d8b717e30-tls-cert\") pod \"2a7b506d-3af4-4790-978f-6a3d8b717e30\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " Jul 23 18:18:43.257817 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.256874 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvnrp\" (UniqueName: \"kubernetes.io/projected/2a7b506d-3af4-4790-978f-6a3d8b717e30-kube-api-access-nvnrp\") pod \"2a7b506d-3af4-4790-978f-6a3d8b717e30\" (UID: \"2a7b506d-3af4-4790-978f-6a3d8b717e30\") " Jul 23 18:18:43.261205 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.261157 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a7b506d-3af4-4790-978f-6a3d8b717e30-kube-api-access-nvnrp" (OuterVolumeSpecName: "kube-api-access-nvnrp") pod "2a7b506d-3af4-4790-978f-6a3d8b717e30" (UID: "2a7b506d-3af4-4790-978f-6a3d8b717e30"). InnerVolumeSpecName "kube-api-access-nvnrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:18:43.271562 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.271520 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a7b506d-3af4-4790-978f-6a3d8b717e30-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2a7b506d-3af4-4790-978f-6a3d8b717e30" (UID: "2a7b506d-3af4-4790-978f-6a3d8b717e30"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:18:43.357969 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.357929 2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a7b506d-3af4-4790-978f-6a3d8b717e30-tls-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:18:43.357969 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.357966 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvnrp\" (UniqueName: \"kubernetes.io/projected/2a7b506d-3af4-4790-978f-6a3d8b717e30-kube-api-access-nvnrp\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:18:43.908127 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.908093 2555 generic.go:356] "Generic (PLEG): container finished" podID="2a7b506d-3af4-4790-978f-6a3d8b717e30" containerID="9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f" exitCode=0 Jul 23 18:18:43.908567 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.908159 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566759f877-9dxzm" Jul 23 18:18:43.908567 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.908189 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566759f877-9dxzm" event={"ID":"2a7b506d-3af4-4790-978f-6a3d8b717e30","Type":"ContainerDied","Data":"9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f"} Jul 23 18:18:43.908567 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.908232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566759f877-9dxzm" event={"ID":"2a7b506d-3af4-4790-978f-6a3d8b717e30","Type":"ContainerDied","Data":"a69af024a66ff647bdf6e97c23a091e87997248eb2d8da7d9d217b1dc037eb83"} Jul 23 18:18:43.908567 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.908251 2555 scope.go:117] "RemoveContainer" containerID="9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f" Jul 23 18:18:43.919560 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.919541 2555 scope.go:117] "RemoveContainer" containerID="9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f" Jul 23 18:18:43.919870 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:18:43.919848 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f\": container with ID starting with 9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f not found: ID does not exist" containerID="9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f" Jul 23 18:18:43.919950 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.919879 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f"} err="failed to get container status \"9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f\": rpc error: code = NotFound desc = could not find container \"9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f\": container with ID starting with 9ee79ae37cba0fb31c55dfd585fc9bce70d971a373c0f7934a0f70ac1706829f not found: ID does not exist" Jul 23 18:18:43.932937 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.932897 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-566759f877-9dxzm"] Jul 23 18:18:43.937946 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:43.937909 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-566759f877-9dxzm"] Jul 23 18:18:44.251882 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:18:44.251784 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a7b506d-3af4-4790-978f-6a3d8b717e30" path="/var/lib/kubelet/pods/2a7b506d-3af4-4790-978f-6a3d8b717e30/volumes" Jul 23 18:19:28.989910 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:28.989870 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f469bfccf-4d72b"] Jul 23 18:19:28.990504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:28.990085 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f469bfccf-4d72b" podUID="b436aad2-51df-4389-9058-d6568746b604" containerName="manager" containerID="cri-o://a5113c48563338a1830e2a523596d453cdbcc77da8ebca8292d063a56a3b124e" gracePeriod=30 Jul 23 18:19:29.149803 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.149768 2555 generic.go:356] "Generic (PLEG): container finished" podID="b436aad2-51df-4389-9058-d6568746b604" containerID="a5113c48563338a1830e2a523596d453cdbcc77da8ebca8292d063a56a3b124e" exitCode=0 Jul 23 18:19:29.150008 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.149846 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f469bfccf-4d72b" event={"ID":"b436aad2-51df-4389-9058-d6568746b604","Type":"ContainerDied","Data":"a5113c48563338a1830e2a523596d453cdbcc77da8ebca8292d063a56a3b124e"} Jul 23 18:19:29.265915 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.265887 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:19:29.387048 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.387014 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert\") pod \"b436aad2-51df-4389-9058-d6568746b604\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " Jul 23 18:19:29.387228 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.387138 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h9hp\" (UniqueName: \"kubernetes.io/projected/b436aad2-51df-4389-9058-d6568746b604-kube-api-access-9h9hp\") pod \"b436aad2-51df-4389-9058-d6568746b604\" (UID: \"b436aad2-51df-4389-9058-d6568746b604\") " Jul 23 18:19:29.389197 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.389161 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b436aad2-51df-4389-9058-d6568746b604" (UID: "b436aad2-51df-4389-9058-d6568746b604"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:19:29.389551 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.389518 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b436aad2-51df-4389-9058-d6568746b604-kube-api-access-9h9hp" (OuterVolumeSpecName: "kube-api-access-9h9hp") pod "b436aad2-51df-4389-9058-d6568746b604" (UID: "b436aad2-51df-4389-9058-d6568746b604"). InnerVolumeSpecName "kube-api-access-9h9hp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:19:29.488445 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.488401 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h9hp\" (UniqueName: \"kubernetes.io/projected/b436aad2-51df-4389-9058-d6568746b604-kube-api-access-9h9hp\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:19:29.488445 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:29.488436 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b436aad2-51df-4389-9058-d6568746b604-webhook-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:19:30.156584 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:30.156550 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f469bfccf-4d72b" Jul 23 18:19:30.156584 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:30.156562 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f469bfccf-4d72b" event={"ID":"b436aad2-51df-4389-9058-d6568746b604","Type":"ContainerDied","Data":"56d1bcd918ff78921b372fd11bc2fb5b00b1005a1f16c4406528de445fc3a508"} Jul 23 18:19:30.157087 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:30.156629 2555 scope.go:117] "RemoveContainer" containerID="a5113c48563338a1830e2a523596d453cdbcc77da8ebca8292d063a56a3b124e" Jul 23 18:19:30.181771 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:30.181741 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f469bfccf-4d72b"] Jul 23 18:19:30.187126 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:30.187101 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f469bfccf-4d72b"] Jul 23 18:19:30.250016 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:30.249982 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b436aad2-51df-4389-9058-d6568746b604" path="/var/lib/kubelet/pods/b436aad2-51df-4389-9058-d6568746b604/volumes" Jul 23 18:19:42.921606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:19:42.921568 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:20:49.499533 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:20:49.499495 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:20:51.015110 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:20:51.015073 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:31.994703 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:31.994672 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:21:31.995473 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:31.995445 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:21:34.396590 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:34.396552 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:35.883182 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:35.883150 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:41.177517 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:41.177478 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:41.282502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:41.282446 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:41.377948 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:41.377913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:41.584715 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:41.584599 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:47.511339 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:47.511287 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:47.572872 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:47.572832 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:47.630643 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:47.630610 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:21:47.733034 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:21:47.733001 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:25:26.384996 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:26.384950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:25:26.472081 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:26.472042 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:25:48.309121 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:48.309083 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx"] Jul 23 18:25:48.309754 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:48.309296 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" podUID="4156e3da-0483-48f2-8a3d-8b4c0b23ee22" containerName="manager" containerID="cri-o://3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad" gracePeriod=10 Jul 23 18:25:51.771025 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:51.771002 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:25:51.830778 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:51.830693 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m74lz\" (UniqueName: \"kubernetes.io/projected/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-kube-api-access-m74lz\") pod \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " Jul 23 18:25:51.830778 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:51.830745 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-extensions-socket-volume\") pod \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\" (UID: \"4156e3da-0483-48f2-8a3d-8b4c0b23ee22\") " Jul 23 18:25:51.831119 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:51.831096 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4156e3da-0483-48f2-8a3d-8b4c0b23ee22" (UID: "4156e3da-0483-48f2-8a3d-8b4c0b23ee22"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:25:51.832739 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:51.832710 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-kube-api-access-m74lz" (OuterVolumeSpecName: "kube-api-access-m74lz") pod "4156e3da-0483-48f2-8a3d-8b4c0b23ee22" (UID: "4156e3da-0483-48f2-8a3d-8b4c0b23ee22"). InnerVolumeSpecName "kube-api-access-m74lz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:25:51.932293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:51.932254 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m74lz\" (UniqueName: \"kubernetes.io/projected/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-kube-api-access-m74lz\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:25:51.932293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:51.932287 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4156e3da-0483-48f2-8a3d-8b4c0b23ee22-extensions-socket-volume\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:25:52.158484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.158447 2555 generic.go:356] "Generic (PLEG): container finished" podID="4156e3da-0483-48f2-8a3d-8b4c0b23ee22" containerID="3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad" exitCode=0 Jul 23 18:25:52.158654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.158495 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" Jul 23 18:25:52.158654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.158531 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" event={"ID":"4156e3da-0483-48f2-8a3d-8b4c0b23ee22","Type":"ContainerDied","Data":"3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad"} Jul 23 18:25:52.158654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.158581 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx" event={"ID":"4156e3da-0483-48f2-8a3d-8b4c0b23ee22","Type":"ContainerDied","Data":"5544a373edee50501401edaee97fefe15bc11b13f56f5b92b7017d183e896728"} Jul 23 18:25:52.158654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.158603 2555 scope.go:117] "RemoveContainer" containerID="3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad" Jul 23 18:25:52.168537 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.168502 2555 scope.go:117] "RemoveContainer" containerID="3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad" Jul 23 18:25:52.168766 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:25:52.168748 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad\": container with ID starting with 3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad not found: ID does not exist" containerID="3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad" Jul 23 18:25:52.168831 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.168773 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad"} err="failed to get container status \"3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad\": rpc error: code = NotFound desc = could not find container \"3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad\": container with ID starting with 3fb59369a0452f7e2f9eac92d5302f49ce1e5285adbc053953ea5fd5d49a88ad not found: ID does not exist" Jul 23 18:25:52.198835 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.198803 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx"] Jul 23 18:25:52.204798 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.204775 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xgpjx"] Jul 23 18:25:52.252797 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:25:52.252758 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4156e3da-0483-48f2-8a3d-8b4c0b23ee22" path="/var/lib/kubelet/pods/4156e3da-0483-48f2-8a3d-8b4c0b23ee22/volumes" Jul 23 18:26:25.132804 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:25.132763 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:26:32.033665 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:32.033631 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:26:32.034090 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:32.034055 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:26:49.463113 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463073 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz"] Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463491 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a7b506d-3af4-4790-978f-6a3d8b717e30" containerName="authorino" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463505 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a7b506d-3af4-4790-978f-6a3d8b717e30" containerName="authorino" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463531 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4156e3da-0483-48f2-8a3d-8b4c0b23ee22" containerName="manager" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463536 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4156e3da-0483-48f2-8a3d-8b4c0b23ee22" containerName="manager" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463544 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b436aad2-51df-4389-9058-d6568746b604" containerName="manager" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463549 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b436aad2-51df-4389-9058-d6568746b604" containerName="manager" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463605 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a7b506d-3af4-4790-978f-6a3d8b717e30" containerName="authorino" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463613 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b436aad2-51df-4389-9058-d6568746b604" containerName="manager" Jul 23 18:26:49.465561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.463621 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4156e3da-0483-48f2-8a3d-8b4c0b23ee22" containerName="manager" Jul 23 18:26:49.466658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.466640 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.469374 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.469341 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qs55q\"" Jul 23 18:26:49.478873 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.478846 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz"] Jul 23 18:26:49.553686 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.553640 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb8f4\" (UniqueName: \"kubernetes.io/projected/ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685-kube-api-access-wb8f4\") pod \"kuadrant-operator-controller-manager-86878c769f-c8blz\" (UID: \"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.553875 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.553722 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c8blz\" (UID: \"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.655080 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.655040 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb8f4\" (UniqueName: \"kubernetes.io/projected/ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685-kube-api-access-wb8f4\") pod \"kuadrant-operator-controller-manager-86878c769f-c8blz\" (UID: \"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.655262 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.655111 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c8blz\" (UID: \"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.655500 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.655483 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c8blz\" (UID: \"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.669157 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.669118 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb8f4\" (UniqueName: \"kubernetes.io/projected/ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685-kube-api-access-wb8f4\") pod \"kuadrant-operator-controller-manager-86878c769f-c8blz\" (UID: \"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.776909 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.776807 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:49.929267 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:49.929240 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz"] Jul 23 18:26:49.931176 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:26:49.931150 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac19172d_04f3_4f4b_bcf4_fb3f4e6cc685.slice/crio-84e04ef6ee944d62a4e2b7615a523670177e2fc6054024f77a7384f09484e46f WatchSource:0}: Error finding container 84e04ef6ee944d62a4e2b7615a523670177e2fc6054024f77a7384f09484e46f: Status 404 returned error can't find the container with id 84e04ef6ee944d62a4e2b7615a523670177e2fc6054024f77a7384f09484e46f Jul 23 18:26:50.453595 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:50.453556 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" event={"ID":"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685","Type":"ContainerStarted","Data":"5d85947b8231e0a4ed3d06597cfc1810553b25f0f654b543d1451719fb7c0362"} Jul 23 18:26:50.453788 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:50.453622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:26:50.453788 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:50.453640 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" event={"ID":"ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685","Type":"ContainerStarted","Data":"84e04ef6ee944d62a4e2b7615a523670177e2fc6054024f77a7384f09484e46f"} Jul 23 18:26:50.476239 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:50.476179 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" podStartSLOduration=1.476163258 podStartE2EDuration="1.476163258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:26:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:26:50.474005126 +0000 UTC m=+1220.171126202" watchObservedRunningTime="2026-07-23 18:26:50.476163258 +0000 UTC m=+1220.173284333" Jul 23 18:26:59.205664 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:26:59.205623 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:27:00.426813 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:27:00.426776 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:27:01.460906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:27:01.460871 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c8blz" Jul 23 18:28:18.396587 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:28:18.396548 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:42.690554 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:29:42.690516 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:29:55.574811 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:29:55.574766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:00.189665 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.189623 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn"] Jul 23 18:30:00.193239 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.193217 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" Jul 23 18:30:00.200513 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.200493 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:30:00.202628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.202609 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" Jul 23 18:30:00.208571 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.208547 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn"] Jul 23 18:30:00.358006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.357979 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn"] Jul 23 18:30:00.361162 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:30:00.361133 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458b0b22_ecd0_495a_be84_eb5a1fa7c15d.slice/crio-5159d61eb95143ab5d9adcba60a9477d1c83611cd8c0ea99295fe84b5d0be25a WatchSource:0}: Error finding container 5159d61eb95143ab5d9adcba60a9477d1c83611cd8c0ea99295fe84b5d0be25a: Status 404 returned error can't find the container with id 5159d61eb95143ab5d9adcba60a9477d1c83611cd8c0ea99295fe84b5d0be25a Jul 23 18:30:00.363383 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.363347 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:30:00.466178 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:00.466089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" event={"ID":"458b0b22-ecd0-495a-be84-eb5a1fa7c15d","Type":"ContainerStarted","Data":"5159d61eb95143ab5d9adcba60a9477d1c83611cd8c0ea99295fe84b5d0be25a"} Jul 23 18:30:04.488404 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:04.488350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" event={"ID":"458b0b22-ecd0-495a-be84-eb5a1fa7c15d","Type":"ContainerStarted","Data":"581178b0fc06c84e10d1644ccce4fbf0002d6e673c5eca83ec86bacf5777468f"} Jul 23 18:30:04.505921 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:04.505864 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" podStartSLOduration=1.281158226 podStartE2EDuration="4.50584886s" totalImagesPullingTime="3.224690634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:30:04.504333152 +0000 UTC m=+1414.201454226" watchObservedRunningTime="2026-07-23 18:30:04.50584886 +0000 UTC m=+1414.202969948" Jul 23 18:30:05.494941 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:05.494903 2555 generic.go:356] "Generic (PLEG): container finished" podID="458b0b22-ecd0-495a-be84-eb5a1fa7c15d" containerID="581178b0fc06c84e10d1644ccce4fbf0002d6e673c5eca83ec86bacf5777468f" exitCode=0 Jul 23 18:30:05.495332 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:05.494963 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" event={"ID":"458b0b22-ecd0-495a-be84-eb5a1fa7c15d","Type":"ContainerDied","Data":"581178b0fc06c84e10d1644ccce4fbf0002d6e673c5eca83ec86bacf5777468f"} Jul 23 18:30:06.650377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:06.650346 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" Jul 23 18:30:07.512876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:07.512843 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" Jul 23 18:30:07.513058 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:07.512832 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747190-sgwpn" event={"ID":"458b0b22-ecd0-495a-be84-eb5a1fa7c15d","Type":"ContainerDied","Data":"5159d61eb95143ab5d9adcba60a9477d1c83611cd8c0ea99295fe84b5d0be25a"} Jul 23 18:30:07.513058 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:07.512972 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5159d61eb95143ab5d9adcba60a9477d1c83611cd8c0ea99295fe84b5d0be25a" Jul 23 18:30:14.580791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:14.580758 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:15.991133 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:15.991085 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:37.957188 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:37.957147 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:30:49.481769 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:30:49.481732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:31:32.073683 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:31:32.073650 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:31:32.074244 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:31:32.073650 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:33:31.303136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:33:31.303042 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:43.538116 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:33:43.538080 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:43.609590 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:33:43.609562 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:51.179927 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:33:51.179885 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:54.307071 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:33:54.307038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:54.407694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:33:54.407662 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:33:54.775985 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:33:54.775946 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:01.692400 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:01.692343 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:01.795179 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:01.795137 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:01.950734 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:01.950647 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:02.274835 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:02.274750 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:02.580232 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:02.580145 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:04.156561 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:04.156527 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:04.244621 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:04.244569 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:04.334010 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:04.333962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:04.518622 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:04.518541 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:04.861543 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:04.861513 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:04.961981 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:04.961952 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:29.735943 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.735911 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml"] Jul 23 18:34:29.736358 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.736345 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458b0b22-ecd0-495a-be84-eb5a1fa7c15d" containerName="cleanup" Jul 23 18:34:29.736358 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.736359 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="458b0b22-ecd0-495a-be84-eb5a1fa7c15d" containerName="cleanup" Jul 23 18:34:29.736483 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.736470 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="458b0b22-ecd0-495a-be84-eb5a1fa7c15d" containerName="cleanup" Jul 23 18:34:29.739625 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.739608 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" Jul 23 18:34:29.742477 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.742456 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:34:29.750261 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.750186 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml"] Jul 23 18:34:29.895462 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.895426 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2293771c-5163-4ced-9892-bfce3a1ce9f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-640e6635-m7pml\" (UID: \"2293771c-5163-4ced-9892-bfce3a1ce9f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" Jul 23 18:34:29.996592 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.996490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2293771c-5163-4ced-9892-bfce3a1ce9f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-640e6635-m7pml\" (UID: \"2293771c-5163-4ced-9892-bfce3a1ce9f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" Jul 23 18:34:29.997187 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:29.997159 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2293771c-5163-4ced-9892-bfce3a1ce9f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-640e6635-m7pml\" (UID: \"2293771c-5163-4ced-9892-bfce3a1ce9f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" Jul 23 18:34:30.051015 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:30.050973 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" Jul 23 18:34:30.393106 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:30.393079 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml"] Jul 23 18:34:30.395150 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:34:30.395117 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2293771c_5163_4ced_9892_bfce3a1ce9f4.slice/crio-de4649d1bc34ac3b7b8ab6e5f65919d40128a0f12fd6b93bdf97a366b4eca98b WatchSource:0}: Error finding container de4649d1bc34ac3b7b8ab6e5f65919d40128a0f12fd6b93bdf97a366b4eca98b: Status 404 returned error can't find the container with id de4649d1bc34ac3b7b8ab6e5f65919d40128a0f12fd6b93bdf97a366b4eca98b Jul 23 18:34:30.878812 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:30.878772 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" event={"ID":"2293771c-5163-4ced-9892-bfce3a1ce9f4","Type":"ContainerStarted","Data":"bf163a79925351ef267148390674a2322cf595827055a58e8b145b13c452d199"} Jul 23 18:34:30.879195 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:30.878832 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" event={"ID":"2293771c-5163-4ced-9892-bfce3a1ce9f4","Type":"ContainerStarted","Data":"de4649d1bc34ac3b7b8ab6e5f65919d40128a0f12fd6b93bdf97a366b4eca98b"} Jul 23 18:34:30.899000 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:30.898933 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" podStartSLOduration=1.898912434 podStartE2EDuration="1.898912434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:34:30.896858026 +0000 UTC m=+1680.593979118" watchObservedRunningTime="2026-07-23 18:34:30.898912434 +0000 UTC m=+1680.596033509" Jul 23 18:34:31.885191 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:31.885157 2555 generic.go:356] "Generic (PLEG): container finished" podID="2293771c-5163-4ced-9892-bfce3a1ce9f4" containerID="bf163a79925351ef267148390674a2322cf595827055a58e8b145b13c452d199" exitCode=0 Jul 23 18:34:31.885654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:31.885248 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" event={"ID":"2293771c-5163-4ced-9892-bfce3a1ce9f4","Type":"ContainerDied","Data":"bf163a79925351ef267148390674a2322cf595827055a58e8b145b13c452d199"} Jul 23 18:34:33.025327 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:33.025298 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" Jul 23 18:34:33.126474 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:33.126443 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2293771c-5163-4ced-9892-bfce3a1ce9f4-maas-api-service-ca\") pod \"2293771c-5163-4ced-9892-bfce3a1ce9f4\" (UID: \"2293771c-5163-4ced-9892-bfce3a1ce9f4\") " Jul 23 18:34:33.126841 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:33.126813 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2293771c-5163-4ced-9892-bfce3a1ce9f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2293771c-5163-4ced-9892-bfce3a1ce9f4" (UID: "2293771c-5163-4ced-9892-bfce3a1ce9f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:34:33.227839 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:33.227745 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2293771c-5163-4ced-9892-bfce3a1ce9f4-maas-api-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:34:33.897533 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:33.897496 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" event={"ID":"2293771c-5163-4ced-9892-bfce3a1ce9f4","Type":"ContainerDied","Data":"de4649d1bc34ac3b7b8ab6e5f65919d40128a0f12fd6b93bdf97a366b4eca98b"} Jul 23 18:34:33.897533 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:33.897532 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml" Jul 23 18:34:33.897747 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:33.897552 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de4649d1bc34ac3b7b8ab6e5f65919d40128a0f12fd6b93bdf97a366b4eca98b" Jul 23 18:34:40.240386 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:40.240344 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:44.824175 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:44.824144 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:44.837060 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:44.837032 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml"] Jul 23 18:34:44.838934 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:44.838911 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-640e6635-m7pml"] Jul 23 18:34:44.982730 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:44.982699 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:45.224988 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:45.224954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:45.420883 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:45.420839 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:45.521304 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:45.521215 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:45.621040 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:45.621003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:45.721446 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:45.721413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:46.021269 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:46.021238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:46.133626 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:46.133586 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:46.247936 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:46.247903 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2293771c-5163-4ced-9892-bfce3a1ce9f4" path="/var/lib/kubelet/pods/2293771c-5163-4ced-9892-bfce3a1ce9f4/volumes" Jul 23 18:34:48.561762 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:48.561718 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:48.666181 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:48.666143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:49.011005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:49.010967 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:49.168662 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:49.168616 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:34:49.269567 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:34:49.269481 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:13.525621 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.525584 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn"] Jul 23 18:35:13.526064 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.526010 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2293771c-5163-4ced-9892-bfce3a1ce9f4" containerName="revoke-keys" Jul 23 18:35:13.526064 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.526023 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2293771c-5163-4ced-9892-bfce3a1ce9f4" containerName="revoke-keys" Jul 23 18:35:13.526138 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.526112 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2293771c-5163-4ced-9892-bfce3a1ce9f4" containerName="revoke-keys" Jul 23 18:35:13.529100 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.529081 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" Jul 23 18:35:13.531525 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.531501 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:35:13.537230 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.536615 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn"] Jul 23 18:35:13.694931 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.694892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fb619a-3f72-4ce6-96ed-779fb0132685-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn\" (UID: \"e9fb619a-3f72-4ce6-96ed-779fb0132685\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" Jul 23 18:35:13.795791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.795693 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fb619a-3f72-4ce6-96ed-779fb0132685-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn\" (UID: \"e9fb619a-3f72-4ce6-96ed-779fb0132685\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" Jul 23 18:35:13.796318 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.796296 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fb619a-3f72-4ce6-96ed-779fb0132685-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn\" (UID: \"e9fb619a-3f72-4ce6-96ed-779fb0132685\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" Jul 23 18:35:13.841702 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.841664 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" Jul 23 18:35:13.986554 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:13.986522 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn"] Jul 23 18:35:13.988589 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:35:13.988560 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fb619a_3f72_4ce6_96ed_779fb0132685.slice/crio-1b1c6f197be9fc646a465349c5e969fdd5a70b58f35e1282ce58a71788a0337d WatchSource:0}: Error finding container 1b1c6f197be9fc646a465349c5e969fdd5a70b58f35e1282ce58a71788a0337d: Status 404 returned error can't find the container with id 1b1c6f197be9fc646a465349c5e969fdd5a70b58f35e1282ce58a71788a0337d Jul 23 18:35:14.114395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:14.114347 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" event={"ID":"e9fb619a-3f72-4ce6-96ed-779fb0132685","Type":"ContainerStarted","Data":"1b1c6f197be9fc646a465349c5e969fdd5a70b58f35e1282ce58a71788a0337d"} Jul 23 18:35:15.120988 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:15.120954 2555 generic.go:356] "Generic (PLEG): container finished" podID="e9fb619a-3f72-4ce6-96ed-779fb0132685" containerID="439aa0227ac847f1adc7d0d61487197eaa7c4d6f71ac9be478469584d407546c" exitCode=0 Jul 23 18:35:15.121429 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:15.121039 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" event={"ID":"e9fb619a-3f72-4ce6-96ed-779fb0132685","Type":"ContainerDied","Data":"439aa0227ac847f1adc7d0d61487197eaa7c4d6f71ac9be478469584d407546c"} Jul 23 18:35:16.272134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:16.272101 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" Jul 23 18:35:16.318560 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:16.318515 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fb619a-3f72-4ce6-96ed-779fb0132685-maas-api-service-ca\") pod \"e9fb619a-3f72-4ce6-96ed-779fb0132685\" (UID: \"e9fb619a-3f72-4ce6-96ed-779fb0132685\") " Jul 23 18:35:16.318879 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:16.318857 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9fb619a-3f72-4ce6-96ed-779fb0132685-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9fb619a-3f72-4ce6-96ed-779fb0132685" (UID: "e9fb619a-3f72-4ce6-96ed-779fb0132685"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:35:16.419358 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:16.419264 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9fb619a-3f72-4ce6-96ed-779fb0132685-maas-api-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:35:17.133003 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:17.132971 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" Jul 23 18:35:17.133003 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:17.132994 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn" event={"ID":"e9fb619a-3f72-4ce6-96ed-779fb0132685","Type":"ContainerDied","Data":"1b1c6f197be9fc646a465349c5e969fdd5a70b58f35e1282ce58a71788a0337d"} Jul 23 18:35:17.133214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:17.133045 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b1c6f197be9fc646a465349c5e969fdd5a70b58f35e1282ce58a71788a0337d" Jul 23 18:35:24.003554 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:24.003511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:28.614033 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:28.613992 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn"] Jul 23 18:35:28.619459 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:28.619419 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f55b464f-7r5sn"] Jul 23 18:35:28.619848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:28.619825 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:28.781558 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:28.781526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:29.025729 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:29.025647 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:29.290305 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:29.290212 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:29.612261 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:29.612223 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:29.762906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:29.762866 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:30.247912 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:30.247876 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9fb619a-3f72-4ce6-96ed-779fb0132685" path="/var/lib/kubelet/pods/e9fb619a-3f72-4ce6-96ed-779fb0132685/volumes" Jul 23 18:35:31.665750 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:31.665716 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:31.762011 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:31.761975 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:31.861841 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:31.861806 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:31.944595 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:31.944514 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:32.522803 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:32.522765 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:32.622730 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:32.622700 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:35:57.442213 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.442174 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl"] Jul 23 18:35:57.442880 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.442588 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9fb619a-3f72-4ce6-96ed-779fb0132685" containerName="revoke-keys" Jul 23 18:35:57.442880 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.442602 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9fb619a-3f72-4ce6-96ed-779fb0132685" containerName="revoke-keys" Jul 23 18:35:57.442880 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.442728 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9fb619a-3f72-4ce6-96ed-779fb0132685" containerName="revoke-keys" Jul 23 18:35:57.446091 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.446073 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" Jul 23 18:35:57.448473 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.448450 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:35:57.454804 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.454779 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl"] Jul 23 18:35:57.515491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.515453 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24012e74-da5d-493d-8a0b-8a6c1d79923a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-84d59500-4vltl\" (UID: \"24012e74-da5d-493d-8a0b-8a6c1d79923a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" Jul 23 18:35:57.616477 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.616436 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24012e74-da5d-493d-8a0b-8a6c1d79923a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-84d59500-4vltl\" (UID: \"24012e74-da5d-493d-8a0b-8a6c1d79923a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" Jul 23 18:35:57.617059 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.617038 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24012e74-da5d-493d-8a0b-8a6c1d79923a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-84d59500-4vltl\" (UID: \"24012e74-da5d-493d-8a0b-8a6c1d79923a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" Jul 23 18:35:57.757724 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.757615 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" Jul 23 18:35:57.901877 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:57.901853 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl"] Jul 23 18:35:57.904238 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:35:57.904206 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24012e74_da5d_493d_8a0b_8a6c1d79923a.slice/crio-a0cdeeac76fa3107923cf056635c49674c4d8ef91170c673c559e3c4686ddd1d WatchSource:0}: Error finding container a0cdeeac76fa3107923cf056635c49674c4d8ef91170c673c559e3c4686ddd1d: Status 404 returned error can't find the container with id a0cdeeac76fa3107923cf056635c49674c4d8ef91170c673c559e3c4686ddd1d Jul 23 18:35:58.343064 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:58.343030 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" event={"ID":"24012e74-da5d-493d-8a0b-8a6c1d79923a","Type":"ContainerStarted","Data":"0122ee487c4ccb7aca19ad413f2be2fd0930c4be557d9544e42ebefaadd5cb2d"} Jul 23 18:35:58.343266 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:58.343089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" event={"ID":"24012e74-da5d-493d-8a0b-8a6c1d79923a","Type":"ContainerStarted","Data":"a0cdeeac76fa3107923cf056635c49674c4d8ef91170c673c559e3c4686ddd1d"} Jul 23 18:35:58.361855 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:58.361798 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" podStartSLOduration=1.361782017 podStartE2EDuration="1.361782017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:35:58.359735942 +0000 UTC m=+1768.056857054" watchObservedRunningTime="2026-07-23 18:35:58.361782017 +0000 UTC m=+1768.058903091" Jul 23 18:35:59.350928 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:59.350892 2555 generic.go:356] "Generic (PLEG): container finished" podID="24012e74-da5d-493d-8a0b-8a6c1d79923a" containerID="0122ee487c4ccb7aca19ad413f2be2fd0930c4be557d9544e42ebefaadd5cb2d" exitCode=0 Jul 23 18:35:59.351333 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:35:59.350966 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" event={"ID":"24012e74-da5d-493d-8a0b-8a6c1d79923a","Type":"ContainerDied","Data":"0122ee487c4ccb7aca19ad413f2be2fd0930c4be557d9544e42ebefaadd5cb2d"} Jul 23 18:36:00.493709 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:00.493682 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" Jul 23 18:36:00.545770 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:00.545725 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24012e74-da5d-493d-8a0b-8a6c1d79923a-maas-api-service-ca\") pod \"24012e74-da5d-493d-8a0b-8a6c1d79923a\" (UID: \"24012e74-da5d-493d-8a0b-8a6c1d79923a\") " Jul 23 18:36:00.546090 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:00.546062 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24012e74-da5d-493d-8a0b-8a6c1d79923a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24012e74-da5d-493d-8a0b-8a6c1d79923a" (UID: "24012e74-da5d-493d-8a0b-8a6c1d79923a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:36:00.646760 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:00.646679 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24012e74-da5d-493d-8a0b-8a6c1d79923a-maas-api-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:36:01.362092 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:01.362058 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" Jul 23 18:36:01.362274 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:01.362061 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl" event={"ID":"24012e74-da5d-493d-8a0b-8a6c1d79923a","Type":"ContainerDied","Data":"a0cdeeac76fa3107923cf056635c49674c4d8ef91170c673c559e3c4686ddd1d"} Jul 23 18:36:01.362274 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:01.362183 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0cdeeac76fa3107923cf056635c49674c4d8ef91170c673c559e3c4686ddd1d" Jul 23 18:36:08.002245 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:08.002201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:12.534813 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:12.534702 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl"] Jul 23 18:36:12.535205 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:12.534907 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:12.536642 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:12.536613 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-84d59500-4vltl"] Jul 23 18:36:13.709332 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:13.709293 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:13.917996 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:13.917960 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:14.139550 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:14.139516 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:14.247990 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:14.247949 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24012e74-da5d-493d-8a0b-8a6c1d79923a" path="/var/lib/kubelet/pods/24012e74-da5d-493d-8a0b-8a6c1d79923a/volumes" Jul 23 18:36:14.490051 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:14.489963 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:15.011615 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:15.011581 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:15.891207 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:15.891172 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:15.989920 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:15.989887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:16.955969 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:16.955932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:17.058481 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.058452 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:17.212063 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.211970 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn"] Jul 23 18:36:17.212395 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.212360 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24012e74-da5d-493d-8a0b-8a6c1d79923a" containerName="revoke-keys" Jul 23 18:36:17.212543 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.212422 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="24012e74-da5d-493d-8a0b-8a6c1d79923a" containerName="revoke-keys" Jul 23 18:36:17.212628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.212555 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="24012e74-da5d-493d-8a0b-8a6c1d79923a" containerName="revoke-keys" Jul 23 18:36:17.216081 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.216060 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.218483 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.218463 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jnbq7\"" Jul 23 18:36:17.218929 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.218907 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-716245e6-serving-cert\"" Jul 23 18:36:17.226654 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.225263 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn"] Jul 23 18:36:17.263005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.262981 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:17.295604 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.295578 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-openshift-service-ca\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.295769 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.295646 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z9s6\" (UniqueName: \"kubernetes.io/projected/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-kube-api-access-2z9s6\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.295769 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.295708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.396960 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.396921 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-openshift-service-ca\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.397129 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.397022 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z9s6\" (UniqueName: \"kubernetes.io/projected/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-kube-api-access-2z9s6\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.397129 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.397090 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.397269 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:36:17.397208 2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-serving-cert: secret "maas-api-e2e-derive-716245e6-serving-cert" not found Jul 23 18:36:17.397313 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:36:17.397287 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls podName:b1c27b07-345b-453c-9a4c-d7d79f8f1d3d nodeName:}" failed. No retries permitted until 2026-07-23 18:36:17.897265131 +0000 UTC m=+1787.594386186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls") pod "maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" (UID: "b1c27b07-345b-453c-9a4c-d7d79f8f1d3d") : secret "maas-api-e2e-derive-716245e6-serving-cert" not found Jul 23 18:36:17.397758 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.397739 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-openshift-service-ca\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.406141 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.406120 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z9s6\" (UniqueName: \"kubernetes.io/projected/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-kube-api-access-2z9s6\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.495735 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.495651 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:17.856682 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.856642 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:17.901627 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.901594 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.903978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.903951 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls\") pod \"maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:17.954057 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:17.954025 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:18.130000 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:18.129917 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:18.278901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:18.278875 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn"] Jul 23 18:36:18.280983 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:36:18.280957 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c27b07_345b_453c_9a4c_d7d79f8f1d3d.slice/crio-707cb0675c3c497387adb5af7e51737b0fce11f0a2bc16e5e7ece421fbe9b014 WatchSource:0}: Error finding container 707cb0675c3c497387adb5af7e51737b0fce11f0a2bc16e5e7ece421fbe9b014: Status 404 returned error can't find the container with id 707cb0675c3c497387adb5af7e51737b0fce11f0a2bc16e5e7ece421fbe9b014 Jul 23 18:36:18.282282 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:18.282225 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:36:18.453004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:18.452915 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" event={"ID":"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d","Type":"ContainerStarted","Data":"707cb0675c3c497387adb5af7e51737b0fce11f0a2bc16e5e7ece421fbe9b014"} Jul 23 18:36:21.472325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:21.472287 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" event={"ID":"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d","Type":"ContainerStarted","Data":"37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d"} Jul 23 18:36:21.472749 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:21.472350 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:21.493165 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:21.493116 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" podStartSLOduration=2.234513266 podStartE2EDuration="4.493101931s" totalImagesPullingTime="2.258588665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:36:21.490763371 +0000 UTC m=+1791.187884471" watchObservedRunningTime="2026-07-23 18:36:21.493101931 +0000 UTC m=+1791.190223033" Jul 23 18:36:27.483203 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:27.483124 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:32.108545 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:32.108513 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:36:32.109766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:32.109746 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:36:41.775134 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:41.775097 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8"] Jul 23 18:36:41.781211 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:41.781168 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" Jul 23 18:36:41.784393 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:41.784341 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:36:41.799318 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:41.799284 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8"] Jul 23 18:36:41.815423 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:41.815382 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d911d212-f870-4ab7-acfd-21c41717e5d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-716245e6-z7sn8\" (UID: \"d911d212-f870-4ab7-acfd-21c41717e5d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" Jul 23 18:36:41.916761 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:41.916715 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d911d212-f870-4ab7-acfd-21c41717e5d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-716245e6-z7sn8\" (UID: \"d911d212-f870-4ab7-acfd-21c41717e5d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" Jul 23 18:36:41.917354 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:41.917333 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d911d212-f870-4ab7-acfd-21c41717e5d4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-716245e6-z7sn8\" (UID: \"d911d212-f870-4ab7-acfd-21c41717e5d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" Jul 23 18:36:42.092976 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:42.092940 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" Jul 23 18:36:42.235725 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:42.235697 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8"] Jul 23 18:36:42.237748 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:36:42.237719 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd911d212_f870_4ab7_acfd_21c41717e5d4.slice/crio-50f1ec072b3ce1fafd8456eaa40078e5d3f1ee69a97aa9efc9dda7b0146dbee0 WatchSource:0}: Error finding container 50f1ec072b3ce1fafd8456eaa40078e5d3f1ee69a97aa9efc9dda7b0146dbee0: Status 404 returned error can't find the container with id 50f1ec072b3ce1fafd8456eaa40078e5d3f1ee69a97aa9efc9dda7b0146dbee0 Jul 23 18:36:42.597007 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:42.596968 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" event={"ID":"d911d212-f870-4ab7-acfd-21c41717e5d4","Type":"ContainerStarted","Data":"34825e00995d0866525f161a41b3da3278a50443dc901fdf4446550e1420d801"} Jul 23 18:36:42.597189 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:42.597029 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" event={"ID":"d911d212-f870-4ab7-acfd-21c41717e5d4","Type":"ContainerStarted","Data":"50f1ec072b3ce1fafd8456eaa40078e5d3f1ee69a97aa9efc9dda7b0146dbee0"} Jul 23 18:36:42.616066 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:42.616007 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" podStartSLOduration=1.615989698 podStartE2EDuration="1.615989698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:36:42.61367113 +0000 UTC m=+1812.310792218" watchObservedRunningTime="2026-07-23 18:36:42.615989698 +0000 UTC m=+1812.313110772" Jul 23 18:36:43.604533 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:43.604493 2555 generic.go:356] "Generic (PLEG): container finished" podID="d911d212-f870-4ab7-acfd-21c41717e5d4" containerID="34825e00995d0866525f161a41b3da3278a50443dc901fdf4446550e1420d801" exitCode=0 Jul 23 18:36:43.604997 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:43.604555 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" event={"ID":"d911d212-f870-4ab7-acfd-21c41717e5d4","Type":"ContainerDied","Data":"34825e00995d0866525f161a41b3da3278a50443dc901fdf4446550e1420d801"} Jul 23 18:36:44.755436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:44.755407 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" Jul 23 18:36:44.839862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:44.839815 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d911d212-f870-4ab7-acfd-21c41717e5d4-maas-api-service-ca\") pod \"d911d212-f870-4ab7-acfd-21c41717e5d4\" (UID: \"d911d212-f870-4ab7-acfd-21c41717e5d4\") " Jul 23 18:36:44.840163 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:44.840136 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d911d212-f870-4ab7-acfd-21c41717e5d4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d911d212-f870-4ab7-acfd-21c41717e5d4" (UID: "d911d212-f870-4ab7-acfd-21c41717e5d4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:36:44.940913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:44.940820 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d911d212-f870-4ab7-acfd-21c41717e5d4-maas-api-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:36:45.619814 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:45.619774 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" Jul 23 18:36:45.619814 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:45.619806 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8" event={"ID":"d911d212-f870-4ab7-acfd-21c41717e5d4","Type":"ContainerDied","Data":"50f1ec072b3ce1fafd8456eaa40078e5d3f1ee69a97aa9efc9dda7b0146dbee0"} Jul 23 18:36:45.620038 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:45.619857 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50f1ec072b3ce1fafd8456eaa40078e5d3f1ee69a97aa9efc9dda7b0146dbee0" Jul 23 18:36:52.220078 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.220030 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn"] Jul 23 18:36:52.220988 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.220927 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" podUID="b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" containerName="maas-api" containerID="cri-o://37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d" gracePeriod=30 Jul 23 18:36:52.243250 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.243221 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:52.480353 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.480283 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:52.508540 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.508513 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls\") pod \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " Jul 23 18:36:52.508717 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.508565 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-openshift-service-ca\") pod \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " Jul 23 18:36:52.508717 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.508596 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z9s6\" (UniqueName: \"kubernetes.io/projected/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-kube-api-access-2z9s6\") pod \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\" (UID: \"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d\") " Jul 23 18:36:52.508982 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.508955 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" (UID: "b1c27b07-345b-453c-9a4c-d7d79f8f1d3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:36:52.510892 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.510864 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-kube-api-access-2z9s6" (OuterVolumeSpecName: "kube-api-access-2z9s6") pod "b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" (UID: "b1c27b07-345b-453c-9a4c-d7d79f8f1d3d"). InnerVolumeSpecName "kube-api-access-2z9s6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:36:52.511020 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.510867 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" (UID: "b1c27b07-345b-453c-9a4c-d7d79f8f1d3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 18:36:52.609308 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.609254 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-maas-api-tls\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:36:52.609308 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.609298 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-openshift-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:36:52.609308 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.609311 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z9s6\" (UniqueName: \"kubernetes.io/projected/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d-kube-api-access-2z9s6\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:36:52.664848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.664813 2555 generic.go:356] "Generic (PLEG): container finished" podID="b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" containerID="37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d" exitCode=0 Jul 23 18:36:52.665017 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.664872 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" Jul 23 18:36:52.665017 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.664893 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" event={"ID":"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d","Type":"ContainerDied","Data":"37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d"} Jul 23 18:36:52.665017 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.664979 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn" event={"ID":"b1c27b07-345b-453c-9a4c-d7d79f8f1d3d","Type":"ContainerDied","Data":"707cb0675c3c497387adb5af7e51737b0fce11f0a2bc16e5e7ece421fbe9b014"} Jul 23 18:36:52.665017 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.665008 2555 scope.go:117] "RemoveContainer" containerID="37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d" Jul 23 18:36:52.674547 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.674526 2555 scope.go:117] "RemoveContainer" containerID="37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d" Jul 23 18:36:52.674805 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:36:52.674786 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d\": container with ID starting with 37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d not found: ID does not exist" containerID="37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d" Jul 23 18:36:52.674845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.674815 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d"} err="failed to get container status \"37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d\": rpc error: code = NotFound desc = could not find container \"37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d\": container with ID starting with 37618c2c1d4ef2081439c17d1b57a5de14ecdbeee23d5be9eb89cc846c55896d not found: ID does not exist" Jul 23 18:36:52.688686 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.688646 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn"] Jul 23 18:36:52.693824 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:52.693797 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-716245e6-79c586f4d9-bcxnn"] Jul 23 18:36:54.248589 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:54.248553 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" path="/var/lib/kubelet/pods/b1c27b07-345b-453c-9a4c-d7d79f8f1d3d/volumes" Jul 23 18:36:56.857747 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:56.857712 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8"] Jul 23 18:36:56.859459 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:56.859433 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:56.860664 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:56.860642 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-716245e6-z7sn8"] Jul 23 18:36:57.025250 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:57.025213 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:57.278165 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:57.278091 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:57.826399 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:57.826333 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:57.927823 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:57.927790 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:36:58.247488 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:58.247456 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d911d212-f870-4ab7-acfd-21c41717e5d4" path="/var/lib/kubelet/pods/d911d212-f870-4ab7-acfd-21c41717e5d4/volumes" Jul 23 18:36:59.390845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:36:59.390808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:05.912542 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:05.912509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:06.028262 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:06.028233 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:06.127442 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:06.127403 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:06.805976 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:06.805942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:06.906345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:06.906313 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:20.707558 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:20.707525 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:32.605411 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:32.605348 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:39.867312 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:39.867276 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:39.971570 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:39.971525 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:40.068109 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:40.068046 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:51.600186 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:51.600156 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:37:55.969174 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:37:55.969063 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:02.136266 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:02.136231 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:03.007701 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:03.007655 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:03.072806 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:03.072769 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:03.976276 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:03.976245 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:04.140377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:04.140329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:13.735681 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:13.735646 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:15.254695 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:15.254659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:21.999263 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:21.999218 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:22.096839 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:22.096793 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:22.194014 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:22.193930 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:22.293562 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:22.293452 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:23.078646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:23.078586 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:23.178154 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:23.178118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:23.281917 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:23.281884 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:25.435236 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:25.435201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:32.431507 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:32.431478 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:32.540907 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:32.540862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:32.770396 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:32.770281 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:32.868126 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:32.868091 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:34.051857 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:34.051820 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:39.981793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:39.981758 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:40.082572 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:40.082530 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:40.219105 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:40.219065 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:40.552157 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:40.552119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:40.896560 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:40.896524 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:43.383953 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:43.383917 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:43.489666 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:43.489631 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:43.584006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:43.583968 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:43.784800 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:43.784714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:44.088342 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:44.088322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:44.187789 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:44.187754 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:47.868719 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:47.868675 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:48.522241 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:48.522193 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:48.621164 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:48.621125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:48.933512 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:48.933474 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:55.295486 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.295438 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:55.401845 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.401811 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:38:55.654657 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.654614 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw"] Jul 23 18:38:55.655107 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.655086 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d911d212-f870-4ab7-acfd-21c41717e5d4" containerName="revoke-keys" Jul 23 18:38:55.655197 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.655109 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d911d212-f870-4ab7-acfd-21c41717e5d4" containerName="revoke-keys" Jul 23 18:38:55.655197 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.655143 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" containerName="maas-api" Jul 23 18:38:55.655197 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.655152 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" containerName="maas-api" Jul 23 18:38:55.655345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.655276 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d911d212-f870-4ab7-acfd-21c41717e5d4" containerName="revoke-keys" Jul 23 18:38:55.655345 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.655295 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1c27b07-345b-453c-9a4c-d7d79f8f1d3d" containerName="maas-api" Jul 23 18:38:55.658875 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.658849 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" Jul 23 18:38:55.661528 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.661502 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:38:55.666843 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.666690 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw"] Jul 23 18:38:55.846356 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.846313 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c23a4b-cedc-4a63-b4fd-c2a788f1d0df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw\" (UID: \"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" Jul 23 18:38:55.947610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.947513 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c23a4b-cedc-4a63-b4fd-c2a788f1d0df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw\" (UID: \"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" Jul 23 18:38:55.948135 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.948113 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c23a4b-cedc-4a63-b4fd-c2a788f1d0df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw\" (UID: \"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" Jul 23 18:38:55.971507 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:55.971466 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" Jul 23 18:38:56.113019 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:56.112995 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw"] Jul 23 18:38:56.115685 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:38:56.115642 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c23a4b_cedc_4a63_b4fd_c2a788f1d0df.slice/crio-52208e574be86e40a94b9ffcda7ba88785879366d68682d0787865a6eef734e5 WatchSource:0}: Error finding container 52208e574be86e40a94b9ffcda7ba88785879366d68682d0787865a6eef734e5: Status 404 returned error can't find the container with id 52208e574be86e40a94b9ffcda7ba88785879366d68682d0787865a6eef734e5 Jul 23 18:38:56.350396 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:56.350340 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" event={"ID":"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df","Type":"ContainerStarted","Data":"52208e574be86e40a94b9ffcda7ba88785879366d68682d0787865a6eef734e5"} Jul 23 18:38:57.357625 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:57.357586 2555 generic.go:356] "Generic (PLEG): container finished" podID="53c23a4b-cedc-4a63-b4fd-c2a788f1d0df" containerID="6e863d3bbe00e360f0b72e9d6576aa1d6a219baa7253c69f030e542bb5f0d082" exitCode=0 Jul 23 18:38:57.358140 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:57.357683 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" event={"ID":"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df","Type":"ContainerDied","Data":"6e863d3bbe00e360f0b72e9d6576aa1d6a219baa7253c69f030e542bb5f0d082"} Jul 23 18:38:58.507733 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:58.507704 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" Jul 23 18:38:58.567913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:58.567870 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c23a4b-cedc-4a63-b4fd-c2a788f1d0df-maas-api-service-ca\") pod \"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df\" (UID: \"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df\") " Jul 23 18:38:58.568242 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:58.568216 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53c23a4b-cedc-4a63-b4fd-c2a788f1d0df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53c23a4b-cedc-4a63-b4fd-c2a788f1d0df" (UID: "53c23a4b-cedc-4a63-b4fd-c2a788f1d0df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:38:58.669315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:58.669221 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c23a4b-cedc-4a63-b4fd-c2a788f1d0df-maas-api-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:38:59.370489 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:59.370450 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" event={"ID":"53c23a4b-cedc-4a63-b4fd-c2a788f1d0df","Type":"ContainerDied","Data":"52208e574be86e40a94b9ffcda7ba88785879366d68682d0787865a6eef734e5"} Jul 23 18:38:59.370685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:59.370493 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw" Jul 23 18:38:59.370685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:38:59.370513 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52208e574be86e40a94b9ffcda7ba88785879366d68682d0787865a6eef734e5" Jul 23 18:39:05.843089 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:05.843049 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:10.759885 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:10.759833 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:10.776325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:10.776292 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw"] Jul 23 18:39:10.778917 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:10.778888 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b6f3e049-kb2lw"] Jul 23 18:39:11.169306 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:11.169270 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:12.255180 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:12.255141 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c23a4b-cedc-4a63-b4fd-c2a788f1d0df" path="/var/lib/kubelet/pods/53c23a4b-cedc-4a63-b4fd-c2a788f1d0df/volumes" Jul 23 18:39:17.846192 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:17.846149 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:17.921397 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:17.921350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:18.126420 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:18.126286 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:18.400149 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:18.400055 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:18.514835 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:18.514783 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:18.612689 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:18.612654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:18.726648 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:18.726549 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:25.223776 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:25.223690 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:25.324310 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:25.324275 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:25.426439 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:25.426409 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:25.588758 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:25.588719 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:31.824946 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:31.824911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:31.900072 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:31.900039 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:38.657003 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:38.656969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:38.952620 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:38.952517 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:39.389409 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:39.389357 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:41.488681 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:41.488620 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:41.582383 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:41.582347 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:41.769316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:41.769215 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:41.948190 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:41.948153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:42.284214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:42.284180 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:42.384789 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:42.384751 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:45.989762 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:45.989720 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:39:55.917264 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:39:55.917227 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:24.547524 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.547483 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt"] Jul 23 18:40:24.548029 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.547940 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c23a4b-cedc-4a63-b4fd-c2a788f1d0df" containerName="revoke-keys" Jul 23 18:40:24.548029 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.547955 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c23a4b-cedc-4a63-b4fd-c2a788f1d0df" containerName="revoke-keys" Jul 23 18:40:24.548110 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.548059 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c23a4b-cedc-4a63-b4fd-c2a788f1d0df" containerName="revoke-keys" Jul 23 18:40:24.551423 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.551395 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" Jul 23 18:40:24.555064 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.555039 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:40:24.561778 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.561747 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt"] Jul 23 18:40:24.648193 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.648146 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a86721-411f-405c-8a4b-aed7dd060d75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-368d7465-c6nlt\" (UID: \"d9a86721-411f-405c-8a4b-aed7dd060d75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" Jul 23 18:40:24.749571 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.749524 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a86721-411f-405c-8a4b-aed7dd060d75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-368d7465-c6nlt\" (UID: \"d9a86721-411f-405c-8a4b-aed7dd060d75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" Jul 23 18:40:24.750184 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.750162 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a86721-411f-405c-8a4b-aed7dd060d75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-368d7465-c6nlt\" (UID: \"d9a86721-411f-405c-8a4b-aed7dd060d75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" Jul 23 18:40:24.864123 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:24.864083 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" Jul 23 18:40:25.014576 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:25.014546 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt"] Jul 23 18:40:25.018098 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:40:25.018063 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a86721_411f_405c_8a4b_aed7dd060d75.slice/crio-4d1fef623510121ee0d9fd3f49d6b1b2b14c008fd9361d5a90a53d6bd0cd9d7b WatchSource:0}: Error finding container 4d1fef623510121ee0d9fd3f49d6b1b2b14c008fd9361d5a90a53d6bd0cd9d7b: Status 404 returned error can't find the container with id 4d1fef623510121ee0d9fd3f49d6b1b2b14c008fd9361d5a90a53d6bd0cd9d7b Jul 23 18:40:25.850164 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:25.850127 2555 generic.go:356] "Generic (PLEG): container finished" podID="d9a86721-411f-405c-8a4b-aed7dd060d75" containerID="dd4aa70935ec533093aa897b01438b3930d1ec698ada0e1b5b38a4c9923b9a4b" exitCode=0 Jul 23 18:40:25.850573 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:25.850182 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" event={"ID":"d9a86721-411f-405c-8a4b-aed7dd060d75","Type":"ContainerDied","Data":"dd4aa70935ec533093aa897b01438b3930d1ec698ada0e1b5b38a4c9923b9a4b"} Jul 23 18:40:25.850573 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:25.850232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" event={"ID":"d9a86721-411f-405c-8a4b-aed7dd060d75","Type":"ContainerStarted","Data":"4d1fef623510121ee0d9fd3f49d6b1b2b14c008fd9361d5a90a53d6bd0cd9d7b"} Jul 23 18:40:26.998994 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:26.998962 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" Jul 23 18:40:27.173040 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:27.172933 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a86721-411f-405c-8a4b-aed7dd060d75-maas-api-service-ca\") pod \"d9a86721-411f-405c-8a4b-aed7dd060d75\" (UID: \"d9a86721-411f-405c-8a4b-aed7dd060d75\") " Jul 23 18:40:27.173289 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:27.173265 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a86721-411f-405c-8a4b-aed7dd060d75-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9a86721-411f-405c-8a4b-aed7dd060d75" (UID: "d9a86721-411f-405c-8a4b-aed7dd060d75"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:40:27.274418 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:27.274356 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a86721-411f-405c-8a4b-aed7dd060d75-maas-api-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:40:27.862541 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:27.862512 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" Jul 23 18:40:27.862541 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:27.862517 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt" event={"ID":"d9a86721-411f-405c-8a4b-aed7dd060d75","Type":"ContainerDied","Data":"4d1fef623510121ee0d9fd3f49d6b1b2b14c008fd9361d5a90a53d6bd0cd9d7b"} Jul 23 18:40:27.862757 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:27.862561 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d1fef623510121ee0d9fd3f49d6b1b2b14c008fd9361d5a90a53d6bd0cd9d7b" Jul 23 18:40:32.240319 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:32.240285 2555 scope.go:117] "RemoveContainer" containerID="bf163a79925351ef267148390674a2322cf595827055a58e8b145b13c452d199" Jul 23 18:40:40.162208 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:40.162157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:44.638893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:44.638860 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:44.657387 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:44.656936 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt"] Jul 23 18:40:44.663515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:44.663475 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-368d7465-c6nlt"] Jul 23 18:40:46.248253 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:46.248219 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a86721-411f-405c-8a4b-aed7dd060d75" path="/var/lib/kubelet/pods/d9a86721-411f-405c-8a4b-aed7dd060d75/volumes" Jul 23 18:40:46.835348 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:46.835312 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:40:51.918326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:40:51.918289 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:16.722399 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:16.722291 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:16.835295 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:16.835258 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:28.892434 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:28.892394 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:30.172623 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:30.172587 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:32.146760 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:32.146728 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:41:32.148889 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:32.148862 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:41:32.261772 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:32.261735 2555 scope.go:117] "RemoveContainer" containerID="439aa0227ac847f1adc7d0d61487197eaa7c4d6f71ac9be478469584d407546c" Jul 23 18:41:36.857432 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:36.857398 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:36.956380 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:36.956322 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:37.164609 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:37.164508 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:37.406059 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:37.406025 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:38.566803 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:38.566770 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:38.660524 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:38.660487 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:38.763056 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:38.763019 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:39.522305 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:39.522269 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:39.622617 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:39.622587 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:39.726384 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:39.726333 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:40.974128 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:40.974094 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:41.199102 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:41.199067 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:48.986110 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:48.986068 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:49.110447 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:49.110404 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:49.755337 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:49.755304 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:57.713270 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:57.713236 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:57.812474 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:57.812444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:41:57.911998 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:41:57.911958 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:08.488827 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:08.488773 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:08.837324 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:08.837223 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:26.896072 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:26.895981 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:29.907526 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:29.907490 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:32.282715 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:32.282609 2555 scope.go:117] "RemoveContainer" containerID="0122ee487c4ccb7aca19ad413f2be2fd0930c4be557d9544e42ebefaadd5cb2d" Jul 23 18:42:37.812104 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:37.812066 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:37.913135 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:37.913082 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:38.187170 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:38.187138 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:38.366024 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:38.365992 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:38.513630 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:38.513514 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:38.878139 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:38.878110 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:41.444739 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:41.444705 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:41.551505 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:41.551467 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:41.644465 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:41.644434 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:41.771322 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:41.771238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:42.136127 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:42.136087 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:42.236700 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:42.236667 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:45.958935 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:45.958898 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:46.294278 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.294181 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:46.727491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.727455 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7"] Jul 23 18:42:46.728131 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.728108 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a86721-411f-405c-8a4b-aed7dd060d75" containerName="revoke-keys" Jul 23 18:42:46.728182 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.728137 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a86721-411f-405c-8a4b-aed7dd060d75" containerName="revoke-keys" Jul 23 18:42:46.728248 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.728237 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a86721-411f-405c-8a4b-aed7dd060d75" containerName="revoke-keys" Jul 23 18:42:46.731534 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.731514 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.735808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.735787 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 18:42:46.735940 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.735924 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d06db7-openshift-default-dockercfg-2qj9t\"" Jul 23 18:42:46.745195 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.745167 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7"] Jul 23 18:42:46.751556 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.751534 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:46.851657 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851625 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-envoy\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.851794 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851664 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-socket\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.851794 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851688 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-data\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.851794 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851751 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-certs\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.852031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-credential-socket\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.852031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851836 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1c816f-6709-44c1-80fd-5822214d13bc-istio-podinfo\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.852031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851882 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79dkq\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-kube-api-access-79dkq\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.852031 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.851965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-istio-token\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.852335 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.852031 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1c816f-6709-44c1-80fd-5822214d13bc-istiod-ca-cert\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952547 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952519 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-istio-token\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1c816f-6709-44c1-80fd-5822214d13bc-istiod-ca-cert\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-envoy\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952646 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-socket\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952662 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-data\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952689 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-certs\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952862 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952815 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-credential-socket\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952969 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952864 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1c816f-6709-44c1-80fd-5822214d13bc-istio-podinfo\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.952969 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.952910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79dkq\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-kube-api-access-79dkq\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.953190 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.953165 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-socket\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.953266 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.953200 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-certs\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.953437 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.953416 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-data\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.953504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.953473 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1c816f-6709-44c1-80fd-5822214d13bc-istiod-ca-cert\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.953504 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.953485 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-credential-socket\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.954902 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.954874 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-envoy\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.955304 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.955282 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1c816f-6709-44c1-80fd-5822214d13bc-istio-podinfo\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.962866 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.962845 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-istio-token\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:46.963258 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:46.963228 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79dkq\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-kube-api-access-79dkq\") pod \"e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:47.042126 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:47.042062 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:47.230710 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:47.230663 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7"] Jul 23 18:42:47.233168 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:42:47.233122 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1c816f_6709_44c1_80fd_5822214d13bc.slice/crio-1df642e817bdedb2e3132e4ae314ad6969690b1292b7d34375fa6170bcf4c13e WatchSource:0}: Error finding container 1df642e817bdedb2e3132e4ae314ad6969690b1292b7d34375fa6170bcf4c13e: Status 404 returned error can't find the container with id 1df642e817bdedb2e3132e4ae314ad6969690b1292b7d34375fa6170bcf4c13e Jul 23 18:42:47.235401 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:47.235382 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:42:47.245841 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:47.245815 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:47.622506 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:47.622471 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" event={"ID":"db1c816f-6709-44c1-80fd-5822214d13bc","Type":"ContainerStarted","Data":"1df642e817bdedb2e3132e4ae314ad6969690b1292b7d34375fa6170bcf4c13e"} Jul 23 18:42:49.761004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:49.760968 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:42:49.761335 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:49.761038 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:42:49.761335 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:49.761067 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:42:50.643176 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:50.643142 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" event={"ID":"db1c816f-6709-44c1-80fd-5822214d13bc","Type":"ContainerStarted","Data":"1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf"} Jul 23 18:42:50.675515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:50.675465 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" podStartSLOduration=2.150316812 podStartE2EDuration="4.675449456s" totalImagesPullingTime="2.525132644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:42:50.674429157 +0000 UTC m=+2180.371550231" watchObservedRunningTime="2026-07-23 18:42:50.675449456 +0000 UTC m=+2180.372570531" Jul 23 18:42:51.042438 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:51.042405 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:51.047062 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:51.047038 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:51.648063 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:51.648024 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:51.649008 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:51.648987 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:42:52.589538 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:52.589504 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:52.688929 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:52.688886 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:52.870766 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:52.870659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:52.965302 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:52.965273 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:53.407123 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:53.407087 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:53.508338 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:53.508293 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:42:57.154576 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:42:57.154533 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:03.294098 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:03.294063 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:03.391863 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:03.391828 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:04.723698 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:04.723661 2555 generic.go:356] "Generic (PLEG): container finished" podID="db1c816f-6709-44c1-80fd-5822214d13bc" containerID="1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf" exitCode=0 Jul 23 18:43:04.724249 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:04.723735 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" event={"ID":"db1c816f-6709-44c1-80fd-5822214d13bc","Type":"ContainerDied","Data":"1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf"} Jul 23 18:43:04.724249 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:04.724167 2555 scope.go:117] "RemoveContainer" containerID="1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf" Jul 23 18:43:04.726274 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:04.726238 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:43:04.726361 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:04.726316 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:43:04.726361 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:04.726345 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:43:05.736676 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:05.736613 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" event={"ID":"db1c816f-6709-44c1-80fd-5822214d13bc","Type":"ContainerStarted","Data":"ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a"} Jul 23 18:43:06.043354 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:06.043249 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:43:06.044598 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:06.044565 2555 patch_prober.go:28] interesting pod/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 23 18:43:06.044756 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:06.044618 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 23 18:43:06.232574 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:06.232442 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:06.648066 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:06.648022 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:43:07.042796 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:07.042693 2555 patch_prober.go:28] interesting pod/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 23 18:43:07.042796 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:07.042754 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 23 18:43:08.043426 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:08.043347 2555 patch_prober.go:28] interesting pod/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 23 18:43:08.043932 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:08.043468 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 23 18:43:09.043053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:09.042552 2555 patch_prober.go:28] interesting pod/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 23 18:43:09.043053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:09.042632 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 23 18:43:10.047616 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:10.047583 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:43:10.048854 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:10.048831 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:43:14.438604 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:14.438559 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:14.538548 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:14.538511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:43:32.304584 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:32.304548 2555 scope.go:117] "RemoveContainer" containerID="34825e00995d0866525f161a41b3da3278a50443dc901fdf4446550e1420d801" Jul 23 18:43:35.729316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:43:35.729280 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:17.411217 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:17.411094 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:18.393241 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.393207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:18.511283 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.511245 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx"] Jul 23 18:44:18.515074 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.515037 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" Jul 23 18:44:18.517636 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.517611 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:44:18.523844 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.523812 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx"] Jul 23 18:44:18.633094 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.633054 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d39346-4a30-4a8f-8c38-348f7d168b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx\" (UID: \"95d39346-4a30-4a8f-8c38-348f7d168b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" Jul 23 18:44:18.734386 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.734275 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d39346-4a30-4a8f-8c38-348f7d168b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx\" (UID: \"95d39346-4a30-4a8f-8c38-348f7d168b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" Jul 23 18:44:18.735000 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.734973 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d39346-4a30-4a8f-8c38-348f7d168b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx\" (UID: \"95d39346-4a30-4a8f-8c38-348f7d168b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" Jul 23 18:44:18.828726 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.828687 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" Jul 23 18:44:18.974379 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:18.974328 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx"] Jul 23 18:44:18.974947 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:44:18.974915 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d39346_4a30_4a8f_8c38_348f7d168b27.slice/crio-d49b268bf0d3f4f39d0c6ef9de564d44a3bd0219dc5b3f1ac679e42a2801782a WatchSource:0}: Error finding container d49b268bf0d3f4f39d0c6ef9de564d44a3bd0219dc5b3f1ac679e42a2801782a: Status 404 returned error can't find the container with id d49b268bf0d3f4f39d0c6ef9de564d44a3bd0219dc5b3f1ac679e42a2801782a Jul 23 18:44:19.139142 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:19.139108 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" event={"ID":"95d39346-4a30-4a8f-8c38-348f7d168b27","Type":"ContainerStarted","Data":"1a79a08df5388b3d3aafd2bc4320659ca03b6517031976104cb9ccaec8c34503"} Jul 23 18:44:19.139316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:19.139161 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" event={"ID":"95d39346-4a30-4a8f-8c38-348f7d168b27","Type":"ContainerStarted","Data":"d49b268bf0d3f4f39d0c6ef9de564d44a3bd0219dc5b3f1ac679e42a2801782a"} Jul 23 18:44:19.159153 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:19.159090 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" podStartSLOduration=1.159073792 podStartE2EDuration="1.159073792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:44:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:44:19.156793049 +0000 UTC m=+2268.853914126" watchObservedRunningTime="2026-07-23 18:44:19.159073792 +0000 UTC m=+2268.856194865" Jul 23 18:44:19.189344 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:19.189308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:19.290220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:19.290127 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:20.146569 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:20.146532 2555 generic.go:356] "Generic (PLEG): container finished" podID="95d39346-4a30-4a8f-8c38-348f7d168b27" containerID="1a79a08df5388b3d3aafd2bc4320659ca03b6517031976104cb9ccaec8c34503" exitCode=0 Jul 23 18:44:20.146982 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:20.146616 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" event={"ID":"95d39346-4a30-4a8f-8c38-348f7d168b27","Type":"ContainerDied","Data":"1a79a08df5388b3d3aafd2bc4320659ca03b6517031976104cb9ccaec8c34503"} Jul 23 18:44:21.289063 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:21.289034 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" Jul 23 18:44:21.460831 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:21.460745 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d39346-4a30-4a8f-8c38-348f7d168b27-maas-api-service-ca\") pod \"95d39346-4a30-4a8f-8c38-348f7d168b27\" (UID: \"95d39346-4a30-4a8f-8c38-348f7d168b27\") " Jul 23 18:44:21.461157 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:21.461136 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95d39346-4a30-4a8f-8c38-348f7d168b27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95d39346-4a30-4a8f-8c38-348f7d168b27" (UID: "95d39346-4a30-4a8f-8c38-348f7d168b27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:44:21.562127 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:21.562088 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d39346-4a30-4a8f-8c38-348f7d168b27-maas-api-service-ca\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:44:22.158901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:22.158867 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" Jul 23 18:44:22.158901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:22.158885 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx" event={"ID":"95d39346-4a30-4a8f-8c38-348f7d168b27","Type":"ContainerDied","Data":"d49b268bf0d3f4f39d0c6ef9de564d44a3bd0219dc5b3f1ac679e42a2801782a"} Jul 23 18:44:22.159123 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:22.158928 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d49b268bf0d3f4f39d0c6ef9de564d44a3bd0219dc5b3f1ac679e42a2801782a" Jul 23 18:44:22.890055 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:22.890017 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:29.070626 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:29.070588 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:33.658405 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:33.658329 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx"] Jul 23 18:44:33.658929 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:33.658464 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:33.660527 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:33.660507 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-47f36d-z98xx"] Jul 23 18:44:34.037267 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:34.037184 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:34.247843 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:34.247808 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95d39346-4a30-4a8f-8c38-348f7d168b27" path="/var/lib/kubelet/pods/95d39346-4a30-4a8f-8c38-348f7d168b27/volumes" Jul 23 18:44:40.623210 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:40.623176 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:40.689277 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:40.689248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:40.913204 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:40.913116 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:41.171214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:41.171126 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:41.884023 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:41.883986 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:41.999016 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:41.998986 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:55.590168 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:55.590132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:44:55.692163 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:44:55.692127 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:00.160615 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.160560 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974"] Jul 23 18:45:00.161115 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.161067 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d39346-4a30-4a8f-8c38-348f7d168b27" containerName="revoke-keys" Jul 23 18:45:00.161115 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.161082 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d39346-4a30-4a8f-8c38-348f7d168b27" containerName="revoke-keys" Jul 23 18:45:00.161235 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.161146 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="95d39346-4a30-4a8f-8c38-348f7d168b27" containerName="revoke-keys" Jul 23 18:45:00.164474 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.164453 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c"] Jul 23 18:45:00.164605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.164587 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974" Jul 23 18:45:00.167643 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.167609 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4sxmg\"" Jul 23 18:45:00.167843 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.167828 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c" Jul 23 18:45:00.176804 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.176778 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974" Jul 23 18:45:00.176990 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.176879 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c"] Jul 23 18:45:00.182000 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.181931 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974"] Jul 23 18:45:00.186672 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.186648 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c" Jul 23 18:45:00.342494 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.342467 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974"] Jul 23 18:45:00.343966 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:45:00.343928 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21507370_a056_4b9b_abea_37c086e4bd91.slice/crio-496ece39e35f100072a4266aa114112d4ed400a483977fd9c8e857d63a91f358 WatchSource:0}: Error finding container 496ece39e35f100072a4266aa114112d4ed400a483977fd9c8e857d63a91f358: Status 404 returned error can't find the container with id 496ece39e35f100072a4266aa114112d4ed400a483977fd9c8e857d63a91f358 Jul 23 18:45:00.367146 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.367110 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974" event={"ID":"21507370-a056-4b9b-abea-37c086e4bd91","Type":"ContainerStarted","Data":"496ece39e35f100072a4266aa114112d4ed400a483977fd9c8e857d63a91f358"} Jul 23 18:45:00.373700 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:00.373477 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c"] Jul 23 18:45:01.374400 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:01.374350 2555 generic.go:356] "Generic (PLEG): container finished" podID="c78b268d-84bc-41e8-8f6a-43082bd6552f" containerID="88370b74021da661f481aa0961683e01c4c93c20bcef117ce9e88af8475eecd0" exitCode=0 Jul 23 18:45:01.374843 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:01.374436 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c" event={"ID":"c78b268d-84bc-41e8-8f6a-43082bd6552f","Type":"ContainerDied","Data":"88370b74021da661f481aa0961683e01c4c93c20bcef117ce9e88af8475eecd0"} Jul 23 18:45:01.374843 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:01.374501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c" event={"ID":"c78b268d-84bc-41e8-8f6a-43082bd6552f","Type":"ContainerStarted","Data":"166b3beac079c81db66e37694c3e28f0940af33569e14584705e13e9c0e587d2"} Jul 23 18:45:01.375899 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:01.375875 2555 generic.go:356] "Generic (PLEG): container finished" podID="21507370-a056-4b9b-abea-37c086e4bd91" containerID="c749909a0de58dd0d27337be34c0a81fc1b4d49a4f2b4ea27b30420ff61d96bd" exitCode=0 Jul 23 18:45:01.376008 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:01.375941 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974" event={"ID":"21507370-a056-4b9b-abea-37c086e4bd91","Type":"ContainerDied","Data":"c749909a0de58dd0d27337be34c0a81fc1b4d49a4f2b4ea27b30420ff61d96bd"} Jul 23 18:45:01.956655 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:01.956614 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:01.984949 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:01.984909 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c"] Jul 23 18:45:02.536966 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:02.536928 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974" Jul 23 18:45:02.560818 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:02.560790 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c" Jul 23 18:45:03.387356 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.387325 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c" Jul 23 18:45:03.387356 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.387338 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c" event={"ID":"c78b268d-84bc-41e8-8f6a-43082bd6552f","Type":"ContainerDied","Data":"166b3beac079c81db66e37694c3e28f0940af33569e14584705e13e9c0e587d2"} Jul 23 18:45:03.387669 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.387423 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="166b3beac079c81db66e37694c3e28f0940af33569e14584705e13e9c0e587d2" Jul 23 18:45:03.388760 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.388740 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974" Jul 23 18:45:03.388869 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.388783 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747205-wn974" event={"ID":"21507370-a056-4b9b-abea-37c086e4bd91","Type":"ContainerDied","Data":"496ece39e35f100072a4266aa114112d4ed400a483977fd9c8e857d63a91f358"} Jul 23 18:45:03.388869 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.388823 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="496ece39e35f100072a4266aa114112d4ed400a483977fd9c8e857d63a91f358" Jul 23 18:45:03.427674 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.427634 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c"] Jul 23 18:45:03.433408 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:03.433349 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d06db7-29747205-ncw7c"] Jul 23 18:45:04.249086 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:04.249048 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c78b268d-84bc-41e8-8f6a-43082bd6552f" path="/var/lib/kubelet/pods/c78b268d-84bc-41e8-8f6a-43082bd6552f/volumes" Jul 23 18:45:06.392950 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:06.392912 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:06.785515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:06.785430 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:13.685584 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:13.685544 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:13.783606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:13.783572 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:13.969453 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:13.969352 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7"] Jul 23 18:45:13.969606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:13.969570 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" containerID="cri-o://ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a" gracePeriod=30 Jul 23 18:45:13.974984 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:13.974961 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:14.240432 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:14.240323 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:14.577219 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:14.577125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:15.002102 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:15.002060 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:15.571749 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:15.571714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:18.293812 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:18.293776 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:18.391428 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:18.391360 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:18.604225 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:18.604190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:18.905921 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:18.905829 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:19.165765 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.165653 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:19.253555 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.253529 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:19.258532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.258513 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:45:19.290161 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290132 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-credential-socket\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290189 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-data\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290210 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-socket\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290232 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-certs\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290259 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-istio-token\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290284 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1c816f-6709-44c1-80fd-5822214d13bc-istiod-ca-cert\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290335 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1c816f-6709-44c1-80fd-5822214d13bc-istio-podinfo\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290450 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:19.290685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290532 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-envoy\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.290685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290548 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:19.290685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290653 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:19.290907 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290800 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db1c816f-6709-44c1-80fd-5822214d13bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:45:19.290968 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.290909 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79dkq\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-kube-api-access-79dkq\") pod \"db1c816f-6709-44c1-80fd-5822214d13bc\" (UID: \"db1c816f-6709-44c1-80fd-5822214d13bc\") " Jul 23 18:45:19.291230 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.291211 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-credential-socket\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.291351 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.291281 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-socket\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.291351 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.291299 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-workload-certs\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.291351 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.291312 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db1c816f-6709-44c1-80fd-5822214d13bc-istiod-ca-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.291351 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.291230 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:19.292830 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.292802 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-kube-api-access-79dkq" (OuterVolumeSpecName: "kube-api-access-79dkq") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "kube-api-access-79dkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:45:19.293006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.292973 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db1c816f-6709-44c1-80fd-5822214d13bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 18:45:19.293086 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.293009 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:45:19.293086 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.293033 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "db1c816f-6709-44c1-80fd-5822214d13bc" (UID: "db1c816f-6709-44c1-80fd-5822214d13bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:45:19.392386 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.392346 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-data\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.392793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.392404 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-istio-token\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.392793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.392417 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db1c816f-6709-44c1-80fd-5822214d13bc-istio-podinfo\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.392793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.392428 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db1c816f-6709-44c1-80fd-5822214d13bc-istio-envoy\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.392793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.392440 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79dkq\" (UniqueName: \"kubernetes.io/projected/db1c816f-6709-44c1-80fd-5822214d13bc-kube-api-access-79dkq\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:45:19.478302 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.478209 2555 generic.go:356] "Generic (PLEG): container finished" podID="db1c816f-6709-44c1-80fd-5822214d13bc" containerID="ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a" exitCode=0 Jul 23 18:45:19.478302 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.478262 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" Jul 23 18:45:19.478302 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.478288 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" event={"ID":"db1c816f-6709-44c1-80fd-5822214d13bc","Type":"ContainerDied","Data":"ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a"} Jul 23 18:45:19.478606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.478343 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7" event={"ID":"db1c816f-6709-44c1-80fd-5822214d13bc","Type":"ContainerDied","Data":"1df642e817bdedb2e3132e4ae314ad6969690b1292b7d34375fa6170bcf4c13e"} Jul 23 18:45:19.478606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.478381 2555 scope.go:117] "RemoveContainer" containerID="ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a" Jul 23 18:45:19.488430 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.488404 2555 scope.go:117] "RemoveContainer" containerID="1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf" Jul 23 18:45:19.496502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.496484 2555 scope.go:117] "RemoveContainer" containerID="ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a" Jul 23 18:45:19.496758 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:45:19.496736 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a\": container with ID starting with ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a not found: ID does not exist" containerID="ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a" Jul 23 18:45:19.496823 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.496766 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a"} err="failed to get container status \"ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a\": rpc error: code = NotFound desc = could not find container \"ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a\": container with ID starting with ca1cde5c5f2b136d4c29a10c84ad97330d67e5e4ef6b9a32765494651a356a6a not found: ID does not exist" Jul 23 18:45:19.496823 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.496783 2555 scope.go:117] "RemoveContainer" containerID="1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf" Jul 23 18:45:19.496993 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:45:19.496978 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf\": container with ID starting with 1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf not found: ID does not exist" containerID="1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf" Jul 23 18:45:19.497030 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.496997 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf"} err="failed to get container status \"1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf\": rpc error: code = NotFound desc = could not find container \"1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf\": container with ID starting with 1348cb1713e85e3dc812d95e1e334630a7e7df5274d245e537f250871482c2cf not found: ID does not exist" Jul 23 18:45:19.504941 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.504919 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7"] Jul 23 18:45:19.506868 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:19.506848 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d06db7-openshift-default-65d45bd97-6h5n7"] Jul 23 18:45:20.247948 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:20.247917 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" path="/var/lib/kubelet/pods/db1c816f-6709-44c1-80fd-5822214d13bc/volumes" Jul 23 18:45:22.810428 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:22.810324 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:23.285110 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.285077 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:23.588348 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588292 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r"] Jul 23 18:45:23.588814 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588794 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21507370-a056-4b9b-abea-37c086e4bd91" containerName="cleanup" Jul 23 18:45:23.588913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588816 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="21507370-a056-4b9b-abea-37c086e4bd91" containerName="cleanup" Jul 23 18:45:23.588913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588842 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" Jul 23 18:45:23.588913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588851 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" Jul 23 18:45:23.588913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588861 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78b268d-84bc-41e8-8f6a-43082bd6552f" containerName="cleanup" Jul 23 18:45:23.588913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588870 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78b268d-84bc-41e8-8f6a-43082bd6552f" containerName="cleanup" Jul 23 18:45:23.588913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588883 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" Jul 23 18:45:23.588913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.588893 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" Jul 23 18:45:23.589254 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.589010 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78b268d-84bc-41e8-8f6a-43082bd6552f" containerName="cleanup" Jul 23 18:45:23.589254 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.589025 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" Jul 23 18:45:23.589254 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.589034 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1c816f-6709-44c1-80fd-5822214d13bc" containerName="istio-proxy" Jul 23 18:45:23.589254 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.589047 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="21507370-a056-4b9b-abea-37c086e4bd91" containerName="cleanup" Jul 23 18:45:23.593590 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.593564 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.603897 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.603872 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 18:45:23.604015 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.603977 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-81e4dc-openshift-default-dockercfg-7bh6c\"" Jul 23 18:45:23.615957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.615913 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r"] Jul 23 18:45:23.630033 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630000 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-token\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630048 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-envoy\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630109 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-certs\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630141 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istiod-ca-cert\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630201 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630179 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-data\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630431 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-credential-socket\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630431 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630293 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-socket\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630431 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630353 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh4r7\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-kube-api-access-kh4r7\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.630431 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.630402 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-podinfo\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.647578 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.647546 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:23.731006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.730964 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-envoy\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731006 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731011 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-certs\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731033 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istiod-ca-cert\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731059 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-data\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731102 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-credential-socket\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731153 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-socket\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731209 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh4r7\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-kube-api-access-kh4r7\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731243 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-podinfo\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.731286 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731281 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-token\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.732660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.732613 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-data\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.732829 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.732747 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-credential-socket\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.733066 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.733040 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istiod-ca-cert\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.733700 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.731943 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-socket\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.734821 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.734746 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-certs\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.735871 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.735848 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-envoy\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.739385 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.739344 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-podinfo\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.756921 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.755208 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-token\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.757570 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.757537 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh4r7\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-kube-api-access-kh4r7\") pod \"e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:23.835492 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.835456 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:23.903621 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:23.903527 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:24.050802 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.050775 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r"] Jul 23 18:45:24.053449 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:45:24.053419 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d2cdac6_c623_411e_9b77_0dc0f8ef152a.slice/crio-61d96a61c3f24dd7e4488f06ffaf8466f5af9002f6ea15d49f72e7f40fdef55a WatchSource:0}: Error finding container 61d96a61c3f24dd7e4488f06ffaf8466f5af9002f6ea15d49f72e7f40fdef55a: Status 404 returned error can't find the container with id 61d96a61c3f24dd7e4488f06ffaf8466f5af9002f6ea15d49f72e7f40fdef55a Jul 23 18:45:24.055524 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.055492 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:45:24.055593 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.055556 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:45:24.055636 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.055590 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:45:24.508707 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.508671 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" event={"ID":"2d2cdac6-c623-411e-9b77-0dc0f8ef152a","Type":"ContainerStarted","Data":"eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793"} Jul 23 18:45:24.508919 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.508724 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" event={"ID":"2d2cdac6-c623-411e-9b77-0dc0f8ef152a","Type":"ContainerStarted","Data":"61d96a61c3f24dd7e4488f06ffaf8466f5af9002f6ea15d49f72e7f40fdef55a"} Jul 23 18:45:24.536462 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.536400 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" podStartSLOduration=1.536378502 podStartE2EDuration="1.536378502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:45:24.533527222 +0000 UTC m=+2334.230648311" watchObservedRunningTime="2026-07-23 18:45:24.536378502 +0000 UTC m=+2334.233499572" Jul 23 18:45:24.904293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:24.904254 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:25.908916 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:25.908883 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:26.520817 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:26.520783 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:26.521711 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:26.521691 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:26.551066 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:26.551027 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:26.656157 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:26.656125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:26.749717 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:26.749683 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:27.284848 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:27.284813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:27.387707 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:27.387673 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:27.488299 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:27.488264 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:31.179206 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:31.179168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:32.336436 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:32.336392 2555 scope.go:117] "RemoveContainer" containerID="6e863d3bbe00e360f0b72e9d6576aa1d6a219baa7253c69f030e542bb5f0d082" Jul 23 18:45:38.593817 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:38.593782 2555 generic.go:356] "Generic (PLEG): container finished" podID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerID="eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793" exitCode=0 Jul 23 18:45:38.594258 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:38.593858 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" event={"ID":"2d2cdac6-c623-411e-9b77-0dc0f8ef152a","Type":"ContainerDied","Data":"eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793"} Jul 23 18:45:38.594258 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:38.594236 2555 scope.go:117] "RemoveContainer" containerID="eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793" Jul 23 18:45:38.595377 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:38.595334 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:45:38.595447 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:38.595434 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:45:38.595502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:38.595485 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:45:39.601743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:39.601703 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" event={"ID":"2d2cdac6-c623-411e-9b77-0dc0f8ef152a","Type":"ContainerStarted","Data":"f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca"} Jul 23 18:45:39.904509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:39.904411 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:39.905744 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:39.905714 2555 patch_prober.go:28] interesting pod/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 23 18:45:39.905895 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:39.905766 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 23 18:45:40.756035 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:40.755995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:40.864116 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:40.864083 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:40.904735 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:40.904700 2555 patch_prober.go:28] interesting pod/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 23 18:45:40.904909 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:40.904755 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 23 18:45:41.520878 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:41.520837 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:41.903939 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:41.903899 2555 patch_prober.go:28] interesting pod/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 23 18:45:41.904360 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:41.903959 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 23 18:45:42.904004 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:42.903962 2555 patch_prober.go:28] interesting pod/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 23 18:45:42.904453 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:42.904023 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 23 18:45:43.908684 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:43.908650 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:43.909863 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:43.909843 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:45:49.909713 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:49.909677 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:50.008546 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:50.008511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:45:56.341786 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:45:56.341748 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:00.092442 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:00.092405 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:00.471206 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:00.471117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:07.183064 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:07.183032 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:07.260975 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:07.260943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:07.465114 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:07.465031 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:07.698893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:07.698860 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:17.577964 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:17.577918 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:17.675682 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:17.675640 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:23.568125 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:23.568088 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:28.067164 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:28.067128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:28.475595 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:28.475559 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:32.184198 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:32.184169 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:46:32.186220 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:32.186186 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:46:32.366564 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:32.366530 2555 scope.go:117] "RemoveContainer" containerID="dd4aa70935ec533093aa897b01438b3930d1ec698ada0e1b5b38a4c9923b9a4b" Jul 23 18:46:35.189169 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:35.189137 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:35.250616 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:35.250585 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:35.447533 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:35.447391 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r"] Jul 23 18:46:35.447714 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:35.447690 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" containerID="cri-o://f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca" gracePeriod=30 Jul 23 18:46:35.471978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:35.471946 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:35.690195 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:35.690157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:36.050581 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:36.050550 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:36.362746 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:36.362715 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:36.579509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:36.579475 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:39.663325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:39.663285 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:39.756830 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:39.756798 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:40.012874 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.012779 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:40.365041 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.365009 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:40.695626 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.695591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:40.712330 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.712304 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:46:40.759554 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759517 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istiod-ca-cert\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759554 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759568 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh4r7\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-kube-api-access-kh4r7\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759613 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-credential-socket\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759641 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-certs\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759664 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-socket\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759702 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-token\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759750 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-envoy\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759781 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-data\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.759836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.759813 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-podinfo\") pod \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\" (UID: \"2d2cdac6-c623-411e-9b77-0dc0f8ef152a\") " Jul 23 18:46:40.760223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760034 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:46:40.760223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760054 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:46:40.760223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760077 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:46:40.760223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760175 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istiod-ca-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.760223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760193 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-credential-socket\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.760223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760209 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-certs\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.760589 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760402 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:46:40.760881 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.760851 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:46:40.762120 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.762095 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-kube-api-access-kh4r7" (OuterVolumeSpecName: "kube-api-access-kh4r7") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "kube-api-access-kh4r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:46:40.762247 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.762211 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:46:40.762461 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.762441 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:46:40.762540 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.762446 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d2cdac6-c623-411e-9b77-0dc0f8ef152a" (UID: "2d2cdac6-c623-411e-9b77-0dc0f8ef152a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 18:46:40.775924 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.775897 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:40.861380 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.861338 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-workload-socket\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.861585 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.861406 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-token\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.861585 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.861422 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-envoy\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.861585 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.861433 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-data\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.861585 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.861447 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-istio-podinfo\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.861585 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.861463 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh4r7\" (UniqueName: \"kubernetes.io/projected/2d2cdac6-c623-411e-9b77-0dc0f8ef152a-kube-api-access-kh4r7\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:46:40.946271 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.946177 2555 generic.go:356] "Generic (PLEG): container finished" podID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerID="f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca" exitCode=0 Jul 23 18:46:40.946271 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.946223 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" Jul 23 18:46:40.946482 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.946267 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" event={"ID":"2d2cdac6-c623-411e-9b77-0dc0f8ef152a","Type":"ContainerDied","Data":"f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca"} Jul 23 18:46:40.946482 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.946319 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r" event={"ID":"2d2cdac6-c623-411e-9b77-0dc0f8ef152a","Type":"ContainerDied","Data":"61d96a61c3f24dd7e4488f06ffaf8466f5af9002f6ea15d49f72e7f40fdef55a"} Jul 23 18:46:40.946482 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.946345 2555 scope.go:117] "RemoveContainer" containerID="f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca" Jul 23 18:46:40.956491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.956471 2555 scope.go:117] "RemoveContainer" containerID="eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793" Jul 23 18:46:40.965241 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.965222 2555 scope.go:117] "RemoveContainer" containerID="f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca" Jul 23 18:46:40.965556 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:46:40.965537 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca\": container with ID starting with f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca not found: ID does not exist" containerID="f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca" Jul 23 18:46:40.965648 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.965564 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca"} err="failed to get container status \"f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca\": rpc error: code = NotFound desc = could not find container \"f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca\": container with ID starting with f48ca2d4ef8fd971d4419973f745ce83f42d67bd453069280b7209229aef54ca not found: ID does not exist" Jul 23 18:46:40.965648 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.965580 2555 scope.go:117] "RemoveContainer" containerID="eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793" Jul 23 18:46:40.965836 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:46:40.965820 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793\": container with ID starting with eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793 not found: ID does not exist" containerID="eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793" Jul 23 18:46:40.965876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.965841 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793"} err="failed to get container status \"eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793\": rpc error: code = NotFound desc = could not find container \"eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793\": container with ID starting with eec7d539ddc927a9e3b07575d725ff7f8348dd7104e6696c6a0270f5f6405793 not found: ID does not exist" Jul 23 18:46:40.971316 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.971291 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r"] Jul 23 18:46:40.974521 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:40.974500 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-81e4dc-openshift-default-54c794fbc5-kz22r"] Jul 23 18:46:42.249100 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:42.249061 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" path="/var/lib/kubelet/pods/2d2cdac6-c623-411e-9b77-0dc0f8ef152a/volumes" Jul 23 18:46:44.148509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.148460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:44.478648 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.478558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:44.760434 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.760307 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk"] Jul 23 18:46:44.760755 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.760739 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" Jul 23 18:46:44.760755 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.760753 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" Jul 23 18:46:44.760912 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.760834 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" Jul 23 18:46:44.760912 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.760842 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" Jul 23 18:46:44.765791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.765764 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.768326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.768305 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 18:46:44.768491 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.768328 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-2fc9f5-openshift-default-dockercfg-n5wcj\"" Jul 23 18:46:44.777136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.777112 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk"] Jul 23 18:46:44.792115 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.792084 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:44.798256 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798225 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-podinfo\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798448 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798267 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-socket\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798448 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798301 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-envoy\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798448 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798337 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsntt\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-kube-api-access-tsntt\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798446 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-data\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798489 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2ee02bd-a592-4165-9882-555e3cccfcf3-istiod-ca-cert\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-token\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798534 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-certs\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.798646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.798579 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-credential-socket\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.899906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.899865 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-data\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.899906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.899913 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2ee02bd-a592-4165-9882-555e3cccfcf3-istiod-ca-cert\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900176 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.899933 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-token\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900176 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.899968 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-certs\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900176 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-credential-socket\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900176 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900039 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-podinfo\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900414 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900186 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-socket\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900414 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900236 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-envoy\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900414 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900294 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsntt\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-kube-api-access-tsntt\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900414 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900332 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-data\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900414 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900391 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-credential-socket\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900706 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900618 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-socket\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900752 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900731 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-certs\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.900876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.900856 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2ee02bd-a592-4165-9882-555e3cccfcf3-istiod-ca-cert\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.902534 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.902509 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-envoy\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.902728 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.902693 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-podinfo\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.910124 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.910092 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-token\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:44.910300 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:44.910280 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsntt\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-kube-api-access-tsntt\") pod \"e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:45.079136 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.079038 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:45.115898 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.115853 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:45.199960 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.199916 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:45.238001 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.237976 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk"] Jul 23 18:46:45.240200 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:46:45.240168 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ee02bd_a592_4165_9882_555e3cccfcf3.slice/crio-8c17519e3816ec362d0d8dba14d35b45c7a993e4006172c4665874851247537f WatchSource:0}: Error finding container 8c17519e3816ec362d0d8dba14d35b45c7a993e4006172c4665874851247537f: Status 404 returned error can't find the container with id 8c17519e3816ec362d0d8dba14d35b45c7a993e4006172c4665874851247537f Jul 23 18:46:45.242685 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.242651 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:46:45.242796 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.242720 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:46:45.242796 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.242749 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:46:45.673210 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.673177 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:45.977333 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.977242 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" event={"ID":"c2ee02bd-a592-4165-9882-555e3cccfcf3","Type":"ContainerStarted","Data":"4e1ff391d54e23a579c17a8a78b4e65a63f84123c009a32f609386ea3f215efa"} Jul 23 18:46:45.977333 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:45.977301 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" event={"ID":"c2ee02bd-a592-4165-9882-555e3cccfcf3","Type":"ContainerStarted","Data":"8c17519e3816ec362d0d8dba14d35b45c7a993e4006172c4665874851247537f"} Jul 23 18:46:46.001357 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:46.001296 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" podStartSLOduration=2.001280565 podStartE2EDuration="2.001280565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:46:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:46:45.999025587 +0000 UTC m=+2415.696146661" watchObservedRunningTime="2026-07-23 18:46:46.001280565 +0000 UTC m=+2415.698401666" Jul 23 18:46:46.079984 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:46.079940 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:46.085407 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:46.085360 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:46.982668 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:46.982642 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:46.983584 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:46.983562 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:46:47.936399 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:47.936352 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:48.037621 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:48.037589 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:48.146275 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:48.146217 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:48.734673 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:48.734642 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:48.937015 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:48.936986 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:49.031019 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:49.030941 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:46:52.442762 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:46:52.442725 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:02.119743 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:02.119701 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": read tcp 10.133.0.2:51730->10.133.0.70:15021: read: connection reset by peer" start-of-body= Jul 23 18:47:02.120198 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:02.119829 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": read tcp 10.133.0.2:51730->10.133.0.70:15021: read: connection reset by peer" Jul 23 18:47:02.208105 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:02.208070 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:02.551780 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:02.551686 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:02.639610 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:02.639576 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:03.080209 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:03.080174 2555 generic.go:356] "Generic (PLEG): container finished" podID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerID="4e1ff391d54e23a579c17a8a78b4e65a63f84123c009a32f609386ea3f215efa" exitCode=0 Jul 23 18:47:03.080506 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:03.080249 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" event={"ID":"c2ee02bd-a592-4165-9882-555e3cccfcf3","Type":"ContainerDied","Data":"4e1ff391d54e23a579c17a8a78b4e65a63f84123c009a32f609386ea3f215efa"} Jul 23 18:47:03.080793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:03.080773 2555 scope.go:117] "RemoveContainer" containerID="4e1ff391d54e23a579c17a8a78b4e65a63f84123c009a32f609386ea3f215efa" Jul 23 18:47:03.081925 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:03.081896 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:47:03.081997 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:03.081984 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:47:03.082035 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:03.082018 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 23 18:47:04.088896 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:04.088855 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" event={"ID":"c2ee02bd-a592-4165-9882-555e3cccfcf3","Type":"ContainerStarted","Data":"354913e2e4ff9a9a8d7ae72675bb8c1357697030afe39bd99480f2125749308b"} Jul 23 18:47:05.079658 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:05.079615 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:47:05.079957 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:05.079932 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 23 18:47:05.080018 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:05.079996 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 23 18:47:06.079978 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:06.079939 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 23 18:47:06.080448 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:06.079999 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 23 18:47:07.079580 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:07.079540 2555 patch_prober.go:28] interesting pod/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 23 18:47:07.079793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:07.079616 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 23 18:47:08.083553 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:08.083514 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:47:08.084061 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:08.083916 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:47:08.084791 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:08.084770 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:47:09.848853 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:09.848817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:09.953941 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:09.953893 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:21.299719 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:21.299688 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:47:39.997002 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:47:39.996965 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:49:26.143385 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:49:26.143320 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:49:27.491005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:49:27.490969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:49:40.931507 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:49:40.931469 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:50:10.719884 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:50:10.719791 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:50:32.421799 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:50:32.421765 2555 scope.go:117] "RemoveContainer" containerID="1a79a08df5388b3d3aafd2bc4320659ca03b6517031976104cb9ccaec8c34503" Jul 23 18:50:51.528507 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:50:51.528475 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:50:57.009801 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:50:57.009765 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:50:57.993214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:50:57.993174 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:06.673052 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:06.673004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:18.385453 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:18.385404 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:18.481571 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:18.481537 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:20.139608 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:20.139570 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:26.078396 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:26.078270 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:51:32.221539 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:32.221501 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:51:32.225195 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:32.225173 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qjdpg_ab021d97-ecab-41fd-8d6b-5e01f0963e4f/ovn-acl-logging/0.log" Jul 23 18:51:32.442432 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:32.442402 2555 scope.go:117] "RemoveContainer" containerID="88370b74021da661f481aa0961683e01c4c93c20bcef117ce9e88af8475eecd0" Jul 23 18:51:38.005214 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:51:38.005177 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:03.559340 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:03.559303 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:03.659271 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:03.659240 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:10.819033 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:10.818989 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:10.920305 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:10.920269 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:32.153694 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:32.153656 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:33.084851 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:33.084813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:34.087455 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.087420 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7722-139933150190384"] Jul 23 18:52:34.087946 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.087827 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" Jul 23 18:52:34.087946 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.087839 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d2cdac6-c623-411e-9b77-0dc0f8ef152a" containerName="istio-proxy" Jul 23 18:52:34.091060 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.091042 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:34.094232 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.094211 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wh5jc\"" Jul 23 18:52:34.100905 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.100881 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150190384"] Jul 23 18:52:34.199532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.199493 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn4lt\" (UniqueName: \"kubernetes.io/projected/768835b4-d1f6-40fd-8371-d4cd97cd8055-kube-api-access-dn4lt\") pod \"test-curl-7722-139933150190384\" (UID: \"768835b4-d1f6-40fd-8371-d4cd97cd8055\") " pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:34.300498 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.300460 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn4lt\" (UniqueName: \"kubernetes.io/projected/768835b4-d1f6-40fd-8371-d4cd97cd8055-kube-api-access-dn4lt\") pod \"test-curl-7722-139933150190384\" (UID: \"768835b4-d1f6-40fd-8371-d4cd97cd8055\") " pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:34.312995 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.312968 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn4lt\" (UniqueName: \"kubernetes.io/projected/768835b4-d1f6-40fd-8371-d4cd97cd8055-kube-api-access-dn4lt\") pod \"test-curl-7722-139933150190384\" (UID: \"768835b4-d1f6-40fd-8371-d4cd97cd8055\") " pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:34.400930 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.400895 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:34.557338 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.557308 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150190384"] Jul 23 18:52:34.558738 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:52:34.558713 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod768835b4_d1f6_40fd_8371_d4cd97cd8055.slice/crio-a592367811c566a625b4183980de1d7f29f32a8f5d686c7de7c6a8838cc909e0 WatchSource:0}: Error finding container a592367811c566a625b4183980de1d7f29f32a8f5d686c7de7c6a8838cc909e0: Status 404 returned error can't find the container with id a592367811c566a625b4183980de1d7f29f32a8f5d686c7de7c6a8838cc909e0 Jul 23 18:52:34.560155 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.560137 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 18:52:34.895819 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:34.895780 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150190384" event={"ID":"768835b4-d1f6-40fd-8371-d4cd97cd8055","Type":"ContainerStarted","Data":"a592367811c566a625b4183980de1d7f29f32a8f5d686c7de7c6a8838cc909e0"} Jul 23 18:52:35.902876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:35.902841 2555 generic.go:356] "Generic (PLEG): container finished" podID="768835b4-d1f6-40fd-8371-d4cd97cd8055" containerID="40bce6940ca08be03ca4954649baa8586d07e9f6e3150ee62808efba253d0787" exitCode=0 Jul 23 18:52:35.903279 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:35.902931 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150190384" event={"ID":"768835b4-d1f6-40fd-8371-d4cd97cd8055","Type":"ContainerDied","Data":"40bce6940ca08be03ca4954649baa8586d07e9f6e3150ee62808efba253d0787"} Jul 23 18:52:37.059326 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.059301 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:37.095233 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.095205 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7722-139933150190384_768835b4-d1f6-40fd-8371-d4cd97cd8055/test-curl-7722-139933150190384/0.log" Jul 23 18:52:37.128300 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.128264 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7722-139933150190384"] Jul 23 18:52:37.134416 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.134358 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7722-139933150190384"] Jul 23 18:52:37.228109 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.228020 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn4lt\" (UniqueName: \"kubernetes.io/projected/768835b4-d1f6-40fd-8371-d4cd97cd8055-kube-api-access-dn4lt\") pod \"768835b4-d1f6-40fd-8371-d4cd97cd8055\" (UID: \"768835b4-d1f6-40fd-8371-d4cd97cd8055\") " Jul 23 18:52:37.230551 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.230515 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768835b4-d1f6-40fd-8371-d4cd97cd8055-kube-api-access-dn4lt" (OuterVolumeSpecName: "kube-api-access-dn4lt") pod "768835b4-d1f6-40fd-8371-d4cd97cd8055" (UID: "768835b4-d1f6-40fd-8371-d4cd97cd8055"). InnerVolumeSpecName "kube-api-access-dn4lt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:52:37.259009 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.258976 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7722-139933150190256"] Jul 23 18:52:37.259388 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.259361 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768835b4-d1f6-40fd-8371-d4cd97cd8055" containerName="test-curl-7722-139933150190384" Jul 23 18:52:37.259443 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.259389 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="768835b4-d1f6-40fd-8371-d4cd97cd8055" containerName="test-curl-7722-139933150190384" Jul 23 18:52:37.259486 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.259477 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="768835b4-d1f6-40fd-8371-d4cd97cd8055" containerName="test-curl-7722-139933150190384" Jul 23 18:52:37.262601 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.262586 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190256" Jul 23 18:52:37.271633 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.271607 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150190256"] Jul 23 18:52:37.329317 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.329281 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn4lt\" (UniqueName: \"kubernetes.io/projected/768835b4-d1f6-40fd-8371-d4cd97cd8055-kube-api-access-dn4lt\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:52:37.429931 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.429892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gbt2\" (UniqueName: \"kubernetes.io/projected/fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3-kube-api-access-5gbt2\") pod \"test-curl-7722-139933150190256\" (UID: \"fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3\") " pod="opendatahub/test-curl-7722-139933150190256" Jul 23 18:52:37.531502 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.531406 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gbt2\" (UniqueName: \"kubernetes.io/projected/fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3-kube-api-access-5gbt2\") pod \"test-curl-7722-139933150190256\" (UID: \"fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3\") " pod="opendatahub/test-curl-7722-139933150190256" Jul 23 18:52:37.559523 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.559483 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gbt2\" (UniqueName: \"kubernetes.io/projected/fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3-kube-api-access-5gbt2\") pod \"test-curl-7722-139933150190256\" (UID: \"fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3\") " pod="opendatahub/test-curl-7722-139933150190256" Jul 23 18:52:37.572458 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.572424 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190256" Jul 23 18:52:37.725880 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.725855 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150190256"] Jul 23 18:52:37.728232 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:52:37.728198 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe9e30a6_a0f5_418a_9a9c_bedc4c9d74f3.slice/crio-057bb66075e7875828e1fa2b1801e48dba82f87d3129b562e82a00ec4eb0580d WatchSource:0}: Error finding container 057bb66075e7875828e1fa2b1801e48dba82f87d3129b562e82a00ec4eb0580d: Status 404 returned error can't find the container with id 057bb66075e7875828e1fa2b1801e48dba82f87d3129b562e82a00ec4eb0580d Jul 23 18:52:37.913927 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.913875 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:37.913927 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.913901 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a592367811c566a625b4183980de1d7f29f32a8f5d686c7de7c6a8838cc909e0" Jul 23 18:52:37.915308 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:37.915280 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150190256" event={"ID":"fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3","Type":"ContainerStarted","Data":"057bb66075e7875828e1fa2b1801e48dba82f87d3129b562e82a00ec4eb0580d"} Jul 23 18:52:37.926286 ip-10-0-128-128 kubenswrapper[2555]: E0723 18:52:37.926257 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7722-139933150190384\" is forbidden: User \"system:node:ip-10-0-128-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-128.ec2.internal' and this object" podUID="768835b4-d1f6-40fd-8371-d4cd97cd8055" pod="opendatahub/test-curl-7722-139933150190384" Jul 23 18:52:38.248532 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:38.248497 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="768835b4-d1f6-40fd-8371-d4cd97cd8055" path="/var/lib/kubelet/pods/768835b4-d1f6-40fd-8371-d4cd97cd8055/volumes" Jul 23 18:52:38.922704 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:38.922667 2555 generic.go:356] "Generic (PLEG): container finished" podID="fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3" containerID="a4b7937d5756c156beacd9ddabca6340d81a9431a14905a1143f2797728b32f3" exitCode=0 Jul 23 18:52:38.922871 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:38.922756 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150190256" event={"ID":"fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3","Type":"ContainerDied","Data":"a4b7937d5756c156beacd9ddabca6340d81a9431a14905a1143f2797728b32f3"} Jul 23 18:52:40.073641 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.073617 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190256" Jul 23 18:52:40.100284 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.100264 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7722-139933150190256_fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3/test-curl-7722-139933150190256/0.log" Jul 23 18:52:40.134572 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.134536 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7722-139933150190256"] Jul 23 18:52:40.137773 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.137748 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7722-139933150190256"] Jul 23 18:52:40.156230 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.156207 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gbt2\" (UniqueName: \"kubernetes.io/projected/fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3-kube-api-access-5gbt2\") pod \"fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3\" (UID: \"fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3\") " Jul 23 18:52:40.158307 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.158276 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3-kube-api-access-5gbt2" (OuterVolumeSpecName: "kube-api-access-5gbt2") pod "fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3" (UID: "fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3"). InnerVolumeSpecName "kube-api-access-5gbt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:52:40.248527 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.248446 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3" path="/var/lib/kubelet/pods/fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3/volumes" Jul 23 18:52:40.257833 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.257812 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gbt2\" (UniqueName: \"kubernetes.io/projected/fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3-kube-api-access-5gbt2\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:52:40.307649 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.307615 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:40.308076 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.308062 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3" containerName="test-curl-7722-139933150190256" Jul 23 18:52:40.308130 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.308077 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3" containerName="test-curl-7722-139933150190256" Jul 23 18:52:40.308167 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.308145 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9e30a6-a0f5-418a-9a9c-bedc4c9d74f3" containerName="test-curl-7722-139933150190256" Jul 23 18:52:40.311224 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.311208 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:40.319971 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.319945 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:40.458858 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.458828 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lppr\" (UniqueName: \"kubernetes.io/projected/e92c3501-4f0a-42a7-9dee-abecdd48a986-kube-api-access-2lppr\") pod \"test-curl-7722-139933150385760\" (UID: \"e92c3501-4f0a-42a7-9dee-abecdd48a986\") " pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:40.559405 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.559287 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lppr\" (UniqueName: \"kubernetes.io/projected/e92c3501-4f0a-42a7-9dee-abecdd48a986-kube-api-access-2lppr\") pod \"test-curl-7722-139933150385760\" (UID: \"e92c3501-4f0a-42a7-9dee-abecdd48a986\") " pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:40.573678 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.573653 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lppr\" (UniqueName: \"kubernetes.io/projected/e92c3501-4f0a-42a7-9dee-abecdd48a986-kube-api-access-2lppr\") pod \"test-curl-7722-139933150385760\" (UID: \"e92c3501-4f0a-42a7-9dee-abecdd48a986\") " pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:40.621549 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.621511 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:40.765705 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.765671 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:40.766601 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:52:40.766573 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode92c3501_4f0a_42a7_9dee_abecdd48a986.slice/crio-e0cb567f1b4a6fb9cb84d3aa55c0d847a2decdc441d7c49b0ae03c256136b796 WatchSource:0}: Error finding container e0cb567f1b4a6fb9cb84d3aa55c0d847a2decdc441d7c49b0ae03c256136b796: Status 404 returned error can't find the container with id e0cb567f1b4a6fb9cb84d3aa55c0d847a2decdc441d7c49b0ae03c256136b796 Jul 23 18:52:40.936208 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.936175 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150190256" Jul 23 18:52:40.936208 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.936216 2555 scope.go:117] "RemoveContainer" containerID="a4b7937d5756c156beacd9ddabca6340d81a9431a14905a1143f2797728b32f3" Jul 23 18:52:40.937718 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:40.937621 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150385760" event={"ID":"e92c3501-4f0a-42a7-9dee-abecdd48a986","Type":"ContainerStarted","Data":"e0cb567f1b4a6fb9cb84d3aa55c0d847a2decdc441d7c49b0ae03c256136b796"} Jul 23 18:52:41.944712 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:41.944666 2555 generic.go:356] "Generic (PLEG): container finished" podID="e92c3501-4f0a-42a7-9dee-abecdd48a986" containerID="be55adfb7211fe8f3a5f05f6fdc4025b42ad5f48313833454294a97a0010ef24" exitCode=0 Jul 23 18:52:41.945118 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:41.944728 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150385760" event={"ID":"e92c3501-4f0a-42a7-9dee-abecdd48a986","Type":"ContainerDied","Data":"be55adfb7211fe8f3a5f05f6fdc4025b42ad5f48313833454294a97a0010ef24"} Jul 23 18:52:43.092748 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.092726 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:43.118173 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.118148 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7722-139933150385760_e92c3501-4f0a-42a7-9dee-abecdd48a986/test-curl-7722-139933150385760/0.log" Jul 23 18:52:43.146210 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.146173 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:43.151010 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.150982 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:43.184747 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.184715 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lppr\" (UniqueName: \"kubernetes.io/projected/e92c3501-4f0a-42a7-9dee-abecdd48a986-kube-api-access-2lppr\") pod \"e92c3501-4f0a-42a7-9dee-abecdd48a986\" (UID: \"e92c3501-4f0a-42a7-9dee-abecdd48a986\") " Jul 23 18:52:43.186849 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.186824 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92c3501-4f0a-42a7-9dee-abecdd48a986-kube-api-access-2lppr" (OuterVolumeSpecName: "kube-api-access-2lppr") pod "e92c3501-4f0a-42a7-9dee-abecdd48a986" (UID: "e92c3501-4f0a-42a7-9dee-abecdd48a986"). InnerVolumeSpecName "kube-api-access-2lppr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:52:43.269194 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.269107 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:43.269589 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.269575 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e92c3501-4f0a-42a7-9dee-abecdd48a986" containerName="test-curl-7722-139933150385760" Jul 23 18:52:43.269643 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.269593 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92c3501-4f0a-42a7-9dee-abecdd48a986" containerName="test-curl-7722-139933150385760" Jul 23 18:52:43.269684 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.269662 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e92c3501-4f0a-42a7-9dee-abecdd48a986" containerName="test-curl-7722-139933150385760" Jul 23 18:52:43.272784 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.272762 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:43.278464 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.278437 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:43.286495 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.286469 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lppr\" (UniqueName: \"kubernetes.io/projected/e92c3501-4f0a-42a7-9dee-abecdd48a986-kube-api-access-2lppr\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:52:43.387040 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.386993 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k8np\" (UniqueName: \"kubernetes.io/projected/60014546-abba-4466-95d8-3e4f313cc631-kube-api-access-9k8np\") pod \"test-curl-7722-139933150385760\" (UID: \"60014546-abba-4466-95d8-3e4f313cc631\") " pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:43.488052 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.488016 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k8np\" (UniqueName: \"kubernetes.io/projected/60014546-abba-4466-95d8-3e4f313cc631-kube-api-access-9k8np\") pod \"test-curl-7722-139933150385760\" (UID: \"60014546-abba-4466-95d8-3e4f313cc631\") " pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:43.506058 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.506019 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k8np\" (UniqueName: \"kubernetes.io/projected/60014546-abba-4466-95d8-3e4f313cc631-kube-api-access-9k8np\") pod \"test-curl-7722-139933150385760\" (UID: \"60014546-abba-4466-95d8-3e4f313cc631\") " pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:43.584034 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.583935 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:43.729202 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.729167 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:43.731100 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:52:43.731072 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60014546_abba_4466_95d8_3e4f313cc631.slice/crio-ea765a7910bf3af1900176d54513fdb03b74efcf1248e3f7983996264b7b366c WatchSource:0}: Error finding container ea765a7910bf3af1900176d54513fdb03b74efcf1248e3f7983996264b7b366c: Status 404 returned error can't find the container with id ea765a7910bf3af1900176d54513fdb03b74efcf1248e3f7983996264b7b366c Jul 23 18:52:43.957064 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.957035 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:43.957260 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.957079 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0cb567f1b4a6fb9cb84d3aa55c0d847a2decdc441d7c49b0ae03c256136b796" Jul 23 18:52:43.966048 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.966001 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150385760" event={"ID":"60014546-abba-4466-95d8-3e4f313cc631","Type":"ContainerStarted","Data":"ea765a7910bf3af1900176d54513fdb03b74efcf1248e3f7983996264b7b366c"} Jul 23 18:52:43.973992 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:43.973959 2555 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7722-139933150385760" oldPodUID="e92c3501-4f0a-42a7-9dee-abecdd48a986" podUID="60014546-abba-4466-95d8-3e4f313cc631" Jul 23 18:52:44.249179 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:44.249091 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e92c3501-4f0a-42a7-9dee-abecdd48a986" path="/var/lib/kubelet/pods/e92c3501-4f0a-42a7-9dee-abecdd48a986/volumes" Jul 23 18:52:44.973273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:44.973237 2555 generic.go:356] "Generic (PLEG): container finished" podID="60014546-abba-4466-95d8-3e4f313cc631" containerID="1f18d961fc84c8a5f2d3ac0b3cf722572aae221242fe861192e82be0c14502d2" exitCode=0 Jul 23 18:52:44.973468 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:44.973312 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150385760" event={"ID":"60014546-abba-4466-95d8-3e4f313cc631","Type":"ContainerDied","Data":"1f18d961fc84c8a5f2d3ac0b3cf722572aae221242fe861192e82be0c14502d2"} Jul 23 18:52:46.130076 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.130049 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:46.153830 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.153799 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7722-139933150385760_60014546-abba-4466-95d8-3e4f313cc631/test-curl-7722-139933150385760/0.log" Jul 23 18:52:46.180063 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.180032 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:46.184696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.184664 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7722-139933150385760"] Jul 23 18:52:46.304568 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.304490 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:46.304963 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.304949 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60014546-abba-4466-95d8-3e4f313cc631" containerName="test-curl-7722-139933150385760" Jul 23 18:52:46.305005 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.304966 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="60014546-abba-4466-95d8-3e4f313cc631" containerName="test-curl-7722-139933150385760" Jul 23 18:52:46.305065 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.305055 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="60014546-abba-4466-95d8-3e4f313cc631" containerName="test-curl-7722-139933150385760" Jul 23 18:52:46.307913 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.307897 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:46.313246 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.313217 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k8np\" (UniqueName: \"kubernetes.io/projected/60014546-abba-4466-95d8-3e4f313cc631-kube-api-access-9k8np\") pod \"60014546-abba-4466-95d8-3e4f313cc631\" (UID: \"60014546-abba-4466-95d8-3e4f313cc631\") " Jul 23 18:52:46.316222 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.316190 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60014546-abba-4466-95d8-3e4f313cc631-kube-api-access-9k8np" (OuterVolumeSpecName: "kube-api-access-9k8np") pod "60014546-abba-4466-95d8-3e4f313cc631" (UID: "60014546-abba-4466-95d8-3e4f313cc631"). InnerVolumeSpecName "kube-api-access-9k8np". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:52:46.321359 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.318140 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:46.414344 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.414294 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dmx4\" (UniqueName: \"kubernetes.io/projected/d7aa9e48-ef92-466f-ad6a-513493763519-kube-api-access-9dmx4\") pod \"test-curl-7722-139933150242256\" (UID: \"d7aa9e48-ef92-466f-ad6a-513493763519\") " pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:46.414555 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.414494 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k8np\" (UniqueName: \"kubernetes.io/projected/60014546-abba-4466-95d8-3e4f313cc631-kube-api-access-9k8np\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:52:46.515906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.515870 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dmx4\" (UniqueName: \"kubernetes.io/projected/d7aa9e48-ef92-466f-ad6a-513493763519-kube-api-access-9dmx4\") pod \"test-curl-7722-139933150242256\" (UID: \"d7aa9e48-ef92-466f-ad6a-513493763519\") " pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:46.527709 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.527673 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dmx4\" (UniqueName: \"kubernetes.io/projected/d7aa9e48-ef92-466f-ad6a-513493763519-kube-api-access-9dmx4\") pod \"test-curl-7722-139933150242256\" (UID: \"d7aa9e48-ef92-466f-ad6a-513493763519\") " pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:46.618492 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.618462 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:46.758565 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.758533 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:46.761750 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:52:46.761721 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7aa9e48_ef92_466f_ad6a_513493763519.slice/crio-8c7ec5bc7658363bf36b404bed269ba057119eaf1fd5180df557dfebe57a6069 WatchSource:0}: Error finding container 8c7ec5bc7658363bf36b404bed269ba057119eaf1fd5180df557dfebe57a6069: Status 404 returned error can't find the container with id 8c7ec5bc7658363bf36b404bed269ba057119eaf1fd5180df557dfebe57a6069 Jul 23 18:52:46.986653 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.986573 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150385760" Jul 23 18:52:46.986653 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.986592 2555 scope.go:117] "RemoveContainer" containerID="1f18d961fc84c8a5f2d3ac0b3cf722572aae221242fe861192e82be0c14502d2" Jul 23 18:52:46.987832 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:46.987805 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150242256" event={"ID":"d7aa9e48-ef92-466f-ad6a-513493763519","Type":"ContainerStarted","Data":"8c7ec5bc7658363bf36b404bed269ba057119eaf1fd5180df557dfebe57a6069"} Jul 23 18:52:47.994922 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:47.994889 2555 generic.go:356] "Generic (PLEG): container finished" podID="d7aa9e48-ef92-466f-ad6a-513493763519" containerID="bf25233bb91e81f912d11b16352d877646298c1249db6f1139c1bba5042eb6c4" exitCode=0 Jul 23 18:52:47.995320 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:47.994975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150242256" event={"ID":"d7aa9e48-ef92-466f-ad6a-513493763519","Type":"ContainerDied","Data":"bf25233bb91e81f912d11b16352d877646298c1249db6f1139c1bba5042eb6c4"} Jul 23 18:52:48.248628 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:48.248551 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60014546-abba-4466-95d8-3e4f313cc631" path="/var/lib/kubelet/pods/60014546-abba-4466-95d8-3e4f313cc631/volumes" Jul 23 18:52:49.157509 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.157481 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:49.183745 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.183718 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7722-139933150242256_d7aa9e48-ef92-466f-ad6a-513493763519/test-curl-7722-139933150242256/0.log" Jul 23 18:52:49.211615 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.211575 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:49.216389 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.216348 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:49.239725 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.239688 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dmx4\" (UniqueName: \"kubernetes.io/projected/d7aa9e48-ef92-466f-ad6a-513493763519-kube-api-access-9dmx4\") pod \"d7aa9e48-ef92-466f-ad6a-513493763519\" (UID: \"d7aa9e48-ef92-466f-ad6a-513493763519\") " Jul 23 18:52:49.241817 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.241785 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7aa9e48-ef92-466f-ad6a-513493763519-kube-api-access-9dmx4" (OuterVolumeSpecName: "kube-api-access-9dmx4") pod "d7aa9e48-ef92-466f-ad6a-513493763519" (UID: "d7aa9e48-ef92-466f-ad6a-513493763519"). InnerVolumeSpecName "kube-api-access-9dmx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:52:49.343630 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.343569 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:49.344599 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.344573 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7aa9e48-ef92-466f-ad6a-513493763519" containerName="test-curl-7722-139933150242256" Jul 23 18:52:49.344599 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.344600 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7aa9e48-ef92-466f-ad6a-513493763519" containerName="test-curl-7722-139933150242256" Jul 23 18:52:49.344804 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.344791 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7aa9e48-ef92-466f-ad6a-513493763519" containerName="test-curl-7722-139933150242256" Jul 23 18:52:49.346789 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.346756 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dmx4\" (UniqueName: \"kubernetes.io/projected/d7aa9e48-ef92-466f-ad6a-513493763519-kube-api-access-9dmx4\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:52:49.348886 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.348800 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:49.351993 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.351969 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:49.447742 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.447692 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncgxh\" (UniqueName: \"kubernetes.io/projected/07bb697f-127a-4ebf-80de-a614e69a36d9-kube-api-access-ncgxh\") pod \"test-curl-7722-139933150242256\" (UID: \"07bb697f-127a-4ebf-80de-a614e69a36d9\") " pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:49.548984 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.548949 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgxh\" (UniqueName: \"kubernetes.io/projected/07bb697f-127a-4ebf-80de-a614e69a36d9-kube-api-access-ncgxh\") pod \"test-curl-7722-139933150242256\" (UID: \"07bb697f-127a-4ebf-80de-a614e69a36d9\") " pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:49.558792 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.558757 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncgxh\" (UniqueName: \"kubernetes.io/projected/07bb697f-127a-4ebf-80de-a614e69a36d9-kube-api-access-ncgxh\") pod \"test-curl-7722-139933150242256\" (UID: \"07bb697f-127a-4ebf-80de-a614e69a36d9\") " pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:49.658968 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.658878 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:49.809144 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:49.809104 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:49.812289 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:52:49.812259 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07bb697f_127a_4ebf_80de_a614e69a36d9.slice/crio-2ab169ab84924ae1c62ab78253857bfa9b155f89f3e3f9e227dcaa8f50c669da WatchSource:0}: Error finding container 2ab169ab84924ae1c62ab78253857bfa9b155f89f3e3f9e227dcaa8f50c669da: Status 404 returned error can't find the container with id 2ab169ab84924ae1c62ab78253857bfa9b155f89f3e3f9e227dcaa8f50c669da Jul 23 18:52:50.010754 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:50.010645 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150242256" event={"ID":"07bb697f-127a-4ebf-80de-a614e69a36d9","Type":"ContainerStarted","Data":"2ab169ab84924ae1c62ab78253857bfa9b155f89f3e3f9e227dcaa8f50c669da"} Jul 23 18:52:50.011989 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:50.011963 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c7ec5bc7658363bf36b404bed269ba057119eaf1fd5180df557dfebe57a6069" Jul 23 18:52:50.011989 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:50.011985 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:50.024883 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:50.024851 2555 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7722-139933150242256" oldPodUID="d7aa9e48-ef92-466f-ad6a-513493763519" podUID="07bb697f-127a-4ebf-80de-a614e69a36d9" Jul 23 18:52:50.248459 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:50.248427 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7aa9e48-ef92-466f-ad6a-513493763519" path="/var/lib/kubelet/pods/d7aa9e48-ef92-466f-ad6a-513493763519/volumes" Jul 23 18:52:51.020293 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:51.020256 2555 generic.go:356] "Generic (PLEG): container finished" podID="07bb697f-127a-4ebf-80de-a614e69a36d9" containerID="26610498af8c3ab6359f05947e9716de20d907071aa0b31bf43b19a967a8b294" exitCode=0 Jul 23 18:52:51.020468 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:51.020324 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7722-139933150242256" event={"ID":"07bb697f-127a-4ebf-80de-a614e69a36d9","Type":"ContainerDied","Data":"26610498af8c3ab6359f05947e9716de20d907071aa0b31bf43b19a967a8b294"} Jul 23 18:52:52.170479 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.170454 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:52.196234 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.196201 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7722-139933150242256_07bb697f-127a-4ebf-80de-a614e69a36d9/test-curl-7722-139933150242256/0.log" Jul 23 18:52:52.222477 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.222446 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:52.228038 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.228006 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7722-139933150242256"] Jul 23 18:52:52.274188 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.274158 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncgxh\" (UniqueName: \"kubernetes.io/projected/07bb697f-127a-4ebf-80de-a614e69a36d9-kube-api-access-ncgxh\") pod \"07bb697f-127a-4ebf-80de-a614e69a36d9\" (UID: \"07bb697f-127a-4ebf-80de-a614e69a36d9\") " Jul 23 18:52:52.276202 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.276152 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07bb697f-127a-4ebf-80de-a614e69a36d9-kube-api-access-ncgxh" (OuterVolumeSpecName: "kube-api-access-ncgxh") pod "07bb697f-127a-4ebf-80de-a614e69a36d9" (UID: "07bb697f-127a-4ebf-80de-a614e69a36d9"). InnerVolumeSpecName "kube-api-access-ncgxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:52:52.375353 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.375317 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncgxh\" (UniqueName: \"kubernetes.io/projected/07bb697f-127a-4ebf-80de-a614e69a36d9-kube-api-access-ncgxh\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:52:52.995556 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:52.995509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:53.033118 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:53.033092 2555 scope.go:117] "RemoveContainer" containerID="26610498af8c3ab6359f05947e9716de20d907071aa0b31bf43b19a967a8b294" Jul 23 18:52:53.033118 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:53.033105 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7722-139933150242256" Jul 23 18:52:53.096101 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:53.096072 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:52:54.247971 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:52:54.247939 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07bb697f-127a-4ebf-80de-a614e69a36d9" path="/var/lib/kubelet/pods/07bb697f-127a-4ebf-80de-a614e69a36d9/volumes" Jul 23 18:53:02.936453 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:02.936406 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:07.510680 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:07.510646 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:07.949682 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:07.949644 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:10.859473 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:10.859438 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:14.597518 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:14.597480 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:14.690221 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:14.690194 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:14.911269 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:14.911183 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:15.135292 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:15.135261 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:16.072993 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:16.072963 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:16.163647 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:16.163619 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:25.919933 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:25.919891 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:30.376280 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:30.376244 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:30.798049 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:30.797963 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:38.786485 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:38.786442 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:38.894485 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:38.894430 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:39.156804 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:39.156768 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:39.178510 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:39.178471 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk"] Jul 23 18:53:39.178714 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:39.178688 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" containerID="cri-o://354913e2e4ff9a9a8d7ae72675bb8c1357697030afe39bd99480f2125749308b" gracePeriod=30 Jul 23 18:53:39.392160 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:39.392113 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:39.848626 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:39.848589 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bf6c8c6d-cbpfx_ed9cccda-fdca-454a-ba9c-c1027f75b1d1/authorino/0.log" Jul 23 18:53:44.329730 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.329690 2555 generic.go:356] "Generic (PLEG): container finished" podID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerID="354913e2e4ff9a9a8d7ae72675bb8c1357697030afe39bd99480f2125749308b" exitCode=0 Jul 23 18:53:44.330191 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.329760 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" event={"ID":"c2ee02bd-a592-4165-9882-555e3cccfcf3","Type":"ContainerDied","Data":"354913e2e4ff9a9a8d7ae72675bb8c1357697030afe39bd99480f2125749308b"} Jul 23 18:53:44.330191 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.329834 2555 scope.go:117] "RemoveContainer" containerID="4e1ff391d54e23a579c17a8a78b4e65a63f84123c009a32f609386ea3f215efa" Jul 23 18:53:44.438325 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.438294 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:53:44.452107 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452012 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-podinfo\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452107 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452057 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-socket\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452107 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452090 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-token\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452107 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452107 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-envoy\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452123 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-data\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452163 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsntt\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-kube-api-access-tsntt\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452214 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-credential-socket\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452252 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-certs\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452484 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452296 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2ee02bd-a592-4165-9882-555e3cccfcf3-istiod-ca-cert\") pod \"c2ee02bd-a592-4165-9882-555e3cccfcf3\" (UID: \"c2ee02bd-a592-4165-9882-555e3cccfcf3\") " Jul 23 18:53:44.452783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452543 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:53:44.452783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452620 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:53:44.452783 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452674 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-socket\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.453002 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.452976 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2ee02bd-a592-4165-9882-555e3cccfcf3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 18:53:44.453555 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.453523 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:53:44.453696 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.453599 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:53:44.454649 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.454624 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 18:53:44.454649 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.454645 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:53:44.454846 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.454818 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-kube-api-access-tsntt" (OuterVolumeSpecName: "kube-api-access-tsntt") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "kube-api-access-tsntt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 18:53:44.455034 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.455016 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2ee02bd-a592-4165-9882-555e3cccfcf3" (UID: "c2ee02bd-a592-4165-9882-555e3cccfcf3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 18:53:44.553876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553838 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2ee02bd-a592-4165-9882-555e3cccfcf3-istiod-ca-cert\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.553876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553869 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-podinfo\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.553876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553879 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-token\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.553876 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553888 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-envoy\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.554162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553896 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-istio-data\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.554162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553904 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsntt\" (UniqueName: \"kubernetes.io/projected/c2ee02bd-a592-4165-9882-555e3cccfcf3-kube-api-access-tsntt\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.554162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553913 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-credential-socket\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:44.554162 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:44.553924 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2ee02bd-a592-4165-9882-555e3cccfcf3-workload-certs\") on node \"ip-10-0-128-128.ec2.internal\" DevicePath \"\"" Jul 23 18:53:45.336618 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:45.336589 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" Jul 23 18:53:45.337032 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:45.336609 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk" event={"ID":"c2ee02bd-a592-4165-9882-555e3cccfcf3","Type":"ContainerDied","Data":"8c17519e3816ec362d0d8dba14d35b45c7a993e4006172c4665874851247537f"} Jul 23 18:53:45.337032 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:45.336683 2555 scope.go:117] "RemoveContainer" containerID="354913e2e4ff9a9a8d7ae72675bb8c1357697030afe39bd99480f2125749308b" Jul 23 18:53:45.366148 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:45.366112 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk"] Jul 23 18:53:45.373636 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:45.373606 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-2fc9f5-openshift-default-6975899bb7-tgwbk"] Jul 23 18:53:46.004939 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:46.004911 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wvdqr_b1ec0c80-04e0-4bca-a4d8-b7f6093c36df/manager/0.log" Jul 23 18:53:46.132296 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:46.132261 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4fgds_7730a224-8f32-49c2-9217-7f25304b9f8e/manager/0.log" Jul 23 18:53:46.248814 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:46.248781 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" path="/var/lib/kubelet/pods/c2ee02bd-a592-4165-9882-555e3cccfcf3/volumes" Jul 23 18:53:46.425899 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:46.425872 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-d92g9_091d8ecb-d73b-4a7e-99a1-f53101a0d347/server/0.log" Jul 23 18:53:46.579859 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:46.579829 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c6n28_fe0d6b30-6243-44b6-abc0-8532682e23e1/manager/0.log" Jul 23 18:53:46.876067 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:46.876039 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b5877c86-zczlg_61f5815f-6367-43e9-82c3-cb5e5a14ed5d/copy-manifests/0.log" Jul 23 18:53:46.890814 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:46.890785 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b5877c86-zczlg_61f5815f-6367-43e9-82c3-cb5e5a14ed5d/manager/0.log" Jul 23 18:53:47.551205 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:47.551174 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747190-sgwpn_458b0b22-ecd0-495a-be84-eb5a1fa7c15d/cleanup/0.log" Jul 23 18:53:47.670793 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:47.670757 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747205-wn974_21507370-a056-4b9b-abea-37c086e4bd91/cleanup/0.log" Jul 23 18:53:47.790903 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:47.790874 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-dttrg_0cfc000c-f667-48a6-9bc9-4c1c6ed961f6/postgres/0.log" Jul 23 18:53:49.249976 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:49.249946 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bf6c8c6d-cbpfx_ed9cccda-fdca-454a-ba9c-c1027f75b1d1/authorino/0.log" Jul 23 18:53:49.375994 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:49.375965 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kzhcw_fa9c2cdd-9897-4a1e-a5f7-b73cac95f856/manager/0.log" Jul 23 18:53:49.492386 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:49.492335 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ptrt5_4d232a16-8861-48a9-ad21-d98df7f36f62/manager/0.log" Jul 23 18:53:49.749663 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:49.749633 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fptmb_a3bd66e9-308b-4d5c-a03d-11b11193c4fa/registry-server/0.log" Jul 23 18:53:49.889575 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:49.889543 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c8blz_ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685/manager/0.log" Jul 23 18:53:50.154942 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:50.154911 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v7zvf_35e26d70-e755-4fc6-b8f8-1e739c482867/manager/0.log" Jul 23 18:53:50.790273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:50.790245 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f87ff99bf-8v88m_cb373434-fc26-4b41-bb3c-0bb56dd65624/kube-auth-proxy/0.log" Jul 23 18:53:51.158073 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:51.158044 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vk65d_529670ef-6888-4e7c-8570-d33ba74800d0/payload-pre-processing/0.log" Jul 23 18:53:51.761295 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:51.761260 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wqmll_db2ee93e-bdab-4e5d-bffd-0e36d042fd25/main/0.log" Jul 23 18:53:51.882993 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:51.882963 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-q2x2n_757a3d73-ed8a-4285-9604-f4d595cc4ab3/main/0.log" Jul 23 18:53:58.477303 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.477262 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jc7fz/must-gather-p69mj"] Jul 23 18:53:58.478223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478183 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" Jul 23 18:53:58.478223 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478214 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" Jul 23 18:53:58.478457 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478261 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07bb697f-127a-4ebf-80de-a614e69a36d9" containerName="test-curl-7722-139933150242256" Jul 23 18:53:58.478457 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478271 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="07bb697f-127a-4ebf-80de-a614e69a36d9" containerName="test-curl-7722-139933150242256" Jul 23 18:53:58.478606 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478588 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" Jul 23 18:53:58.478679 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478618 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="07bb697f-127a-4ebf-80de-a614e69a36d9" containerName="test-curl-7722-139933150242256" Jul 23 18:53:58.478679 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478631 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" Jul 23 18:53:58.478864 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478850 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" Jul 23 18:53:58.478922 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.478865 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2ee02bd-a592-4165-9882-555e3cccfcf3" containerName="istio-proxy" Jul 23 18:53:58.486740 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.486712 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.493173 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.492717 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jc7fz\"/\"kube-root-ca.crt\"" Jul 23 18:53:58.493768 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.493739 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jc7fz\"/\"default-dockercfg-zx572\"" Jul 23 18:53:58.493901 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.493774 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jc7fz\"/\"openshift-service-ca.crt\"" Jul 23 18:53:58.495918 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.495892 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc7fz/must-gather-p69mj"] Jul 23 18:53:58.504635 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.504609 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:58.571488 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.571453 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" limit=1024 Jul 23 18:53:58.588722 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.588687 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d286a220-7ac0-4f45-aeb0-42a752b3e4a2-must-gather-output\") pod \"must-gather-p69mj\" (UID: \"d286a220-7ac0-4f45-aeb0-42a752b3e4a2\") " pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.588836 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.588820 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqvtf\" (UniqueName: \"kubernetes.io/projected/d286a220-7ac0-4f45-aeb0-42a752b3e4a2-kube-api-access-tqvtf\") pod \"must-gather-p69mj\" (UID: \"d286a220-7ac0-4f45-aeb0-42a752b3e4a2\") " pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.690315 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.690271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqvtf\" (UniqueName: \"kubernetes.io/projected/d286a220-7ac0-4f45-aeb0-42a752b3e4a2-kube-api-access-tqvtf\") pod \"must-gather-p69mj\" (UID: \"d286a220-7ac0-4f45-aeb0-42a752b3e4a2\") " pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.690499 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.690338 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d286a220-7ac0-4f45-aeb0-42a752b3e4a2-must-gather-output\") pod \"must-gather-p69mj\" (UID: \"d286a220-7ac0-4f45-aeb0-42a752b3e4a2\") " pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.690687 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.690667 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d286a220-7ac0-4f45-aeb0-42a752b3e4a2-must-gather-output\") pod \"must-gather-p69mj\" (UID: \"d286a220-7ac0-4f45-aeb0-42a752b3e4a2\") " pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.700614 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.700580 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqvtf\" (UniqueName: \"kubernetes.io/projected/d286a220-7ac0-4f45-aeb0-42a752b3e4a2-kube-api-access-tqvtf\") pod \"must-gather-p69mj\" (UID: \"d286a220-7ac0-4f45-aeb0-42a752b3e4a2\") " pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.796767 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.796679 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc7fz/must-gather-p69mj" Jul 23 18:53:58.940770 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:58.940740 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc7fz/must-gather-p69mj"] Jul 23 18:53:58.942568 ip-10-0-128-128 kubenswrapper[2555]: W0723 18:53:58.942539 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd286a220_7ac0_4f45_aeb0_42a752b3e4a2.slice/crio-c2148c056c2b47e4b468810fa7d4ea5a655184ae7cdbca4dc975a0bb309f9a92 WatchSource:0}: Error finding container c2148c056c2b47e4b468810fa7d4ea5a655184ae7cdbca4dc975a0bb309f9a92: Status 404 returned error can't find the container with id c2148c056c2b47e4b468810fa7d4ea5a655184ae7cdbca4dc975a0bb309f9a92 Jul 23 18:53:59.414953 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:53:59.414920 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc7fz/must-gather-p69mj" event={"ID":"d286a220-7ac0-4f45-aeb0-42a752b3e4a2","Type":"ContainerStarted","Data":"c2148c056c2b47e4b468810fa7d4ea5a655184ae7cdbca4dc975a0bb309f9a92"} Jul 23 18:54:00.423159 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:00.423126 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc7fz/must-gather-p69mj" event={"ID":"d286a220-7ac0-4f45-aeb0-42a752b3e4a2","Type":"ContainerStarted","Data":"8c1cc83a748367ac5c3cebb4b97b776f2b9c692ca88d744ba38da71b36865b40"} Jul 23 18:54:00.423614 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:00.423170 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc7fz/must-gather-p69mj" event={"ID":"d286a220-7ac0-4f45-aeb0-42a752b3e4a2","Type":"ContainerStarted","Data":"107318fd8a93a01b65ded1e327a2c6c668838d0c4d4f4ef5f46e07f0ba54608e"} Jul 23 18:54:00.447159 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:00.447104 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jc7fz/must-gather-p69mj" podStartSLOduration=1.4564771429999999 podStartE2EDuration="2.447088902s" totalImagesPullingTime="990.611759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:54:00.444136386 +0000 UTC m=+2850.141257488" watchObservedRunningTime="2026-07-23 18:54:00.447088902 +0000 UTC m=+2850.144210268" Jul 23 18:54:01.992062 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:01.992030 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tlbgv_1d8cac9c-1c74-4cc6-9c38-3dd55ed682f4/global-pull-secret-syncer/0.log" Jul 23 18:54:02.078949 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:02.078919 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5n6tt_a56fdb46-cda2-4a61-afbd-5b46df9a57c3/konnectivity-agent/0.log" Jul 23 18:54:02.212646 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:02.212616 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-128.ec2.internal_b519a6809738d51a152b4478c04773cc/haproxy/0.log" Jul 23 18:54:07.362200 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:07.361994 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7bf6c8c6d-cbpfx_ed9cccda-fdca-454a-ba9c-c1027f75b1d1/authorino/0.log" Jul 23 18:54:07.404919 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:07.404882 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kzhcw_fa9c2cdd-9897-4a1e-a5f7-b73cac95f856/manager/0.log" Jul 23 18:54:07.441538 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:07.441507 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ptrt5_4d232a16-8861-48a9-ad21-d98df7f36f62/manager/0.log" Jul 23 18:54:07.552129 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:07.552089 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fptmb_a3bd66e9-308b-4d5c-a03d-11b11193c4fa/registry-server/0.log" Jul 23 18:54:07.763462 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:07.763361 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c8blz_ac19172d-04f3-4f4b-bcf4-fb3f4e6cc685/manager/0.log" Jul 23 18:54:07.881956 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:07.881921 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v7zvf_35e26d70-e755-4fc6-b8f8-1e739c482867/manager/0.log" Jul 23 18:54:09.655230 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:09.655145 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_30b617af-fec5-4698-a93d-e8d21419f2aa/alertmanager/0.log" Jul 23 18:54:09.693154 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:09.693121 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_30b617af-fec5-4698-a93d-e8d21419f2aa/config-reloader/0.log" Jul 23 18:54:09.720605 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:09.720571 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_30b617af-fec5-4698-a93d-e8d21419f2aa/kube-rbac-proxy-web/0.log" Jul 23 18:54:09.748965 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:09.748935 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_30b617af-fec5-4698-a93d-e8d21419f2aa/kube-rbac-proxy/0.log" Jul 23 18:54:09.779086 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:09.779055 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_30b617af-fec5-4698-a93d-e8d21419f2aa/kube-rbac-proxy-metric/0.log" Jul 23 18:54:09.807885 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:09.807829 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_30b617af-fec5-4698-a93d-e8d21419f2aa/prom-label-proxy/0.log" Jul 23 18:54:09.835808 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:09.835778 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_30b617af-fec5-4698-a93d-e8d21419f2aa/init-config-reloader/0.log" Jul 23 18:54:10.032989 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.032895 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fbbd8bfc9-n6mjx_869f2371-88de-4bf7-abe1-c5a1a27c5efb/metrics-server/0.log" Jul 23 18:54:10.069088 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.069059 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-tr8gr_c1ee50f5-e93f-4e63-8aa2-9a61ced0204b/monitoring-plugin/0.log" Jul 23 18:54:10.196903 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.196872 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9bsf9_bd3c7e8f-4252-401d-b14c-adebb8744827/node-exporter/0.log" Jul 23 18:54:10.222154 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.222124 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9bsf9_bd3c7e8f-4252-401d-b14c-adebb8744827/kube-rbac-proxy/0.log" Jul 23 18:54:10.250310 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.250279 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9bsf9_bd3c7e8f-4252-401d-b14c-adebb8744827/init-textfile/0.log" Jul 23 18:54:10.919029 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.918983 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d674c8-jlnhj_ce395276-f347-41ec-b9e8-f7a90a8da900/thanos-query/0.log" Jul 23 18:54:10.945732 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.945681 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d674c8-jlnhj_ce395276-f347-41ec-b9e8-f7a90a8da900/kube-rbac-proxy-web/0.log" Jul 23 18:54:10.972482 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.972435 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d674c8-jlnhj_ce395276-f347-41ec-b9e8-f7a90a8da900/kube-rbac-proxy/0.log" Jul 23 18:54:10.998906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:10.998873 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d674c8-jlnhj_ce395276-f347-41ec-b9e8-f7a90a8da900/prom-label-proxy/0.log" Jul 23 18:54:11.024448 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.024412 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d674c8-jlnhj_ce395276-f347-41ec-b9e8-f7a90a8da900/kube-rbac-proxy-rules/0.log" Jul 23 18:54:11.049962 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.049912 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-768d674c8-jlnhj_ce395276-f347-41ec-b9e8-f7a90a8da900/kube-rbac-proxy-metrics/0.log" Jul 23 18:54:11.722894 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.722851 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr"] Jul 23 18:54:11.728972 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.728936 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.738387 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.738328 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr"] Jul 23 18:54:11.836912 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.836864 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7z7m\" (UniqueName: \"kubernetes.io/projected/f14d1e71-0789-4a75-9e9f-5063bab04a5a-kube-api-access-h7z7m\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.837339 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.837311 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-sys\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.837542 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.837523 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-proc\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.837660 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.837646 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-lib-modules\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.837813 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.837802 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-podres\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.939221 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.939164 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-sys\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.939221 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.939228 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-proc\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.939778 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.939252 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-lib-modules\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.939778 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.939321 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-podres\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.939778 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.939397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7z7m\" (UniqueName: \"kubernetes.io/projected/f14d1e71-0789-4a75-9e9f-5063bab04a5a-kube-api-access-h7z7m\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.939934 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.939900 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-sys\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.939998 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.939967 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-proc\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.940119 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.940088 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-lib-modules\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.940273 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.940250 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f14d1e71-0789-4a75-9e9f-5063bab04a5a-podres\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:11.951827 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:11.951756 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7z7m\" (UniqueName: \"kubernetes.io/projected/f14d1e71-0789-4a75-9e9f-5063bab04a5a-kube-api-access-h7z7m\") pod \"perf-node-gather-daemonset-g5htr\" (UID: \"f14d1e71-0789-4a75-9e9f-5063bab04a5a\") " pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:12.047500 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:12.047405 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:12.237800 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:12.237739 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr"] Jul 23 18:54:12.525926 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:12.525881 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" event={"ID":"f14d1e71-0789-4a75-9e9f-5063bab04a5a","Type":"ContainerStarted","Data":"701fc0f2d56711c25b21123c79599eaf2cf1789df705b7a459fd59f1b64e1cb9"} Jul 23 18:54:12.526154 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:12.526080 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:12.526154 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:12.526132 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" event={"ID":"f14d1e71-0789-4a75-9e9f-5063bab04a5a","Type":"ContainerStarted","Data":"b62014441c158c97bb70716dc1eb29777248a5a92c61402126d97d5fe2dd7743"} Jul 23 18:54:12.547126 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:12.547044 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" podStartSLOduration=1.547020353 podStartE2EDuration="1.547020353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:54:12.542941569 +0000 UTC m=+2862.240062670" watchObservedRunningTime="2026-07-23 18:54:12.547020353 +0000 UTC m=+2862.244141428" Jul 23 18:54:13.465781 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:13.465753 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-857c57d97c-t6m8v_05732ba1-fc4f-4b2c-b474-d5ce0e122f27/console/0.log" Jul 23 18:54:13.521764 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:13.521734 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-5hkdh_f17db44b-991d-4c45-8db3-cea8b82259f1/download-server/0.log" Jul 23 18:54:15.427510 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:15.427461 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9l82s_1ffe51b0-c8c4-470c-8391-13e33dfca268/dns/0.log" Jul 23 18:54:15.456469 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:15.456436 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9l82s_1ffe51b0-c8c4-470c-8391-13e33dfca268/kube-rbac-proxy/0.log" Jul 23 18:54:15.553052 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:15.553025 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2tj8f_7e3d2607-3f4d-4eac-a069-ca8ae834bc85/dns-node-resolver/0.log" Jul 23 18:54:16.221992 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:16.221946 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68ddf7ccf4-6xt8n_d0c5a37a-a36f-4f34-b8e8-cfee6c5c59c3/registry/0.log" Jul 23 18:54:16.361906 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:16.361872 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wss72_1e71b11c-141b-467a-9e9d-deb696739f55/node-ca/0.log" Jul 23 18:54:17.793403 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:17.793359 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f87ff99bf-8v88m_cb373434-fc26-4b41-bb3c-0bb56dd65624/kube-auth-proxy/0.log" Jul 23 18:54:17.971682 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:17.971647 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vk65d_529670ef-6888-4e7c-8570-d33ba74800d0/payload-pre-processing/0.log" Jul 23 18:54:18.541943 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:18.541915 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jc7fz/perf-node-gather-daemonset-g5htr" Jul 23 18:54:18.840070 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:18.840035 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bpdcc_f2aeac23-dcb6-43ba-b1e8-9646614ed07a/serve-healthcheck-canary/0.log" Jul 23 18:54:19.585570 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:19.585536 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sbwl6_720a8d7f-6625-4173-acfa-4c6df41f3519/kube-rbac-proxy/0.log" Jul 23 18:54:19.617249 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:19.617222 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sbwl6_720a8d7f-6625-4173-acfa-4c6df41f3519/exporter/0.log" Jul 23 18:54:19.651515 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:19.651483 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sbwl6_720a8d7f-6625-4173-acfa-4c6df41f3519/extractor/0.log" Jul 23 18:54:22.231762 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:22.231731 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wvdqr_b1ec0c80-04e0-4bca-a4d8-b7f6093c36df/manager/0.log" Jul 23 18:54:22.494720 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:22.494616 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4fgds_7730a224-8f32-49c2-9217-7f25304b9f8e/manager/0.log" Jul 23 18:54:22.738992 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:22.738960 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-d92g9_091d8ecb-d73b-4a7e-99a1-f53101a0d347/server/0.log" Jul 23 18:54:22.843105 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:22.843072 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c6n28_fe0d6b30-6243-44b6-abc0-8532682e23e1/manager/0.log" Jul 23 18:54:23.046053 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:23.046025 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b5877c86-zczlg_61f5815f-6367-43e9-82c3-cb5e5a14ed5d/manager/0.log" Jul 23 18:54:23.071079 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:23.071051 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56b5877c86-zczlg_61f5815f-6367-43e9-82c3-cb5e5a14ed5d/copy-manifests/0.log" Jul 23 18:54:23.089099 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:23.089072 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f87ff99bf-8v88m_cb373434-fc26-4b41-bb3c-0bb56dd65624/kube-auth-proxy/0.log" Jul 23 18:54:23.216129 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:23.216036 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vk65d_529670ef-6888-4e7c-8570-d33ba74800d0/payload-pre-processing/0.log" Jul 23 18:54:24.515498 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:24.515419 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ndnmg_4f39d568-9384-48da-bf7e-039f634d3ee8/openshift-lws-operator/0.log" Jul 23 18:54:24.838991 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:24.838962 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f87ff99bf-8v88m_cb373434-fc26-4b41-bb3c-0bb56dd65624/kube-auth-proxy/0.log" Jul 23 18:54:24.964893 ip-10-0-128-128 kubenswrapper[2555]: I0723 18:54:24.964850 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vk65d_529670ef-6888-4e7c-8570-d33ba74800d0/payload-pre-processing/0.log"