Jul 28 17:55:04.527240 ip-10-0-132-23 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 17:55:04.527254 ip-10-0-132-23 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 17:55:04.527264 ip-10-0-132-23 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 17:55:04.527574 ip-10-0-132-23 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 17:55:14.561722 ip-10-0-132-23 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 17:55:14.561743 ip-10-0-132-23 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8742498c7a5a4466931686faa38704ce --
Jul 28 17:57:05.733197 ip-10-0-132-23 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 17:57:06.160551 ip-10-0-132-23 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:57:06.160551 ip-10-0-132-23 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 17:57:06.160551 ip-10-0-132-23 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:57:06.160551 ip-10-0-132-23 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 17:57:06.160551 ip-10-0-132-23 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:57:06.163460 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.163286    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 17:57:06.169603 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169577    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:57:06.169603 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169602    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:57:06.169603 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169606    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169612    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169615    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169619    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169627    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169631    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169633    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169637    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169640    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169643    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169645    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169648    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169651    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169654    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169657    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169660    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169662    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169666    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169669    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169671    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:57:06.169720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169674    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169677    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169680    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169685    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169689    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169693    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169696    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169699    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169702    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169705    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169708    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169712    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169715    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169719    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169725    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169729    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169733    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169736    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169739    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:57:06.170361 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169743    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169746    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169750    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169753    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169755    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169758    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169761    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169764    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169766    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169769    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169771    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169774    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169777    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169780    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169782    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169785    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169788    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169790    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169793    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169796    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:57:06.171281 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169799    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169801    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169804    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169808    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169828    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169832    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169835    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169839    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169842    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169845    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169850    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169853    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169856    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169859    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169862    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169865    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169868    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169870    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169873    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:57:06.171970 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169876    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169879    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169882    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169885    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169888    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169890    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169893    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169896    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169899    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169902    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169905    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169908    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169912    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169916    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169920    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169923    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169941    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169944    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.169947    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172129    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:57:06.172473 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172146    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172150    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172153    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172157    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172160    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172163    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172166    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172171    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172176    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172180    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172183    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172187    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172190    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172193    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172196    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172199    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172202    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172205    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172207    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:57:06.173020 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172212    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172216    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172219    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172222    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172226    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172229    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172232    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172235    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172238    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172241    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172243    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172247    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172250    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172253    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172256    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172259    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172262    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172266    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172269    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172272    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:57:06.173547 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172275    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172277    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172280    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172283    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172285    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172288    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172291    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172294    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172296    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172299    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172302    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172305    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172308    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172310    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172313    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172315    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172319    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172322    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172324    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172327    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:57:06.174120 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172329    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172332    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172335    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172338    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172341    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172344    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172347    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172350    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172353    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172355    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172358    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172361    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172363    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172366    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172369    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172372    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172375    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172378    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172381    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172383    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:57:06.174647 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172386    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172389    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172391    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172395    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172398    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172400    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172403    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172406    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172409    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172411    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172414    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172416    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172419    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172422    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172425    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172428    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172431    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172435    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172440    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:57:06.175233 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172883    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172889    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172892    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172896    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172899    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172902    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172906    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172909    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172911    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172914    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172917    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172920    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172923    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172926    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172928    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172931    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172952    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172955    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172958    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172961    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:57:06.175738 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172964    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172966    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172970    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172972    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172975    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172978    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172981    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172985    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172988    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172990    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172994    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.172997    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173000    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173003    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173006    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173009    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173012    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173014    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173017    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173020    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:57:06.176293 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173023    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173026    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173030    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173032    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173035    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173038    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173041    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173047    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173051    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173055    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173058    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173061    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173064    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173067    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173070    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173073    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173076    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173079    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173082    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:57:06.176858 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173084    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173089    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173092    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173096    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173099    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173102    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173104    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173108    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173111    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173113    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173116    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173119    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173121    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173124    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173127    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173130    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173132    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173135    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173138    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:57:06.177364 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173140    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173143    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173146    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173148    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173153    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173155    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173158    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173161    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173164    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173167    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173169    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173172    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173176    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173179    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173183    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173186    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173189    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173192    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173195    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:57:06.177897 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173198    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.173201    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173291    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173299    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173308    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173313    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173318    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173322    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173326    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173331    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173334    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173337    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173341    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173345    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173348    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173352    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173355    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173358    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173362    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173365    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173370    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173373    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173376    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173380    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 17:57:06.178421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173383    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173387    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173391    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173395    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173398    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173402    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173405    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173409    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173412    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173416    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173420    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173424    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173427    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173430    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173433    2575 flags.go:64] FLAG: --enable-server="true"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173437    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173442    2575 flags.go:64] FLAG: --event-burst="100"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173445    2575 flags.go:64] FLAG: --event-qps="50"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173448    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173452    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173455    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173459    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173462    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173465    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173469    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 28 17:57:06.179064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173472    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173475    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173480    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173483    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173486    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173489    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173492    2575 flags.go:64] FLAG: --feature-gates=""
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173497    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173500    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173504    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173507    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173510    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173514    2575 flags.go:64] FLAG: --help="false"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173517    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173521    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173524    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173527    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173531    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173535    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173538    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173541    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173544    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173548    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 17:57:06.179736 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173551    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173555    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173558    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173561    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173564    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173568    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173571    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173574    2575 flags.go:64] FLAG: --lock-file=""
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173577    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173580    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173583    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173590    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173594    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173597    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173600    2575 flags.go:64] FLAG: --logging-format="text"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173603    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173607    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173612    2575 flags.go:64] FLAG: --manifest-url=""
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173615    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173620    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173623    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173629    2575 flags.go:64] FLAG: --max-pods="110"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173633    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173636    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173639    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 17:57:06.180453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173643    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173646    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173649    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173652    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173660    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173664    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173667    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173670    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173673    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173680    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173683    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173686    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173690    2575 flags.go:64] FLAG: --port="10250"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173694    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173697    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01cfbcec1058c2d41"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173701    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173704    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173707    2575 flags.go:64] FLAG: --register-node="true"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173710    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173714    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173718    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173721    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173724    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173728    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173733    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 17:57:06.181141 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173736    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173739    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173742    2575 flags.go:64] FLAG: --runonce="false"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173745    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173750    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173753    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173756    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173760    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173763    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173766    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173770    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173773    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173776    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173779    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173782    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173786    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173789    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173792    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173797    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173800    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173803    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173808    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173811    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173814    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173817    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 17:57:06.181845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173820    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173824    2575 flags.go:64] FLAG: --v="2"
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173828    2575 flags.go:64] FLAG: --version="false"
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173834    2575 flags.go:64] FLAG: --vmodule=""
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173838    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.173841    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174015    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174020    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174023    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174026    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174029    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174034    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174037    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174040    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174042    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174045    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174048    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174051    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174054    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174056    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174059    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:57:06.182530 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174062    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174065    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174067    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174070    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174073    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174076    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174079    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174081    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174086    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174089    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174092    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174095    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174098    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174101    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174103    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174107    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174110    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174114    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174116    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174119    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:57:06.183112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174122    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174125    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174130    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174133    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174136    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174139    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174141    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174144    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174147    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174150    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174153    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174155    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174158    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174161    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174166    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174169    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174173    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174176    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174180    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174183    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:57:06.183650 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174186    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174189    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174192    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174195    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174198    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174202    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174204    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174208    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174212    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174216    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174220    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174222    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174225    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174228    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174232    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174235    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174238    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174241    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174244    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:57:06.184326 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174246    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174249    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174252    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174255    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174258    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174261    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174263    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174266    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174269    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174272    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174275    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174277    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174280    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174283    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174286    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174289    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174291    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174294    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174297    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174300    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:57:06.184826 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174302    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:57:06.185366 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174305    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:57:06.185366 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174310    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:57:06.185366 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174313    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:57:06.185366 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.174317    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:57:06.185366 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.175007    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:57:06.185657 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.185630    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 17:57:06.185693 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.185658    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 17:57:06.186465 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.186453    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 17:57:06.186501 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.186466    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 17:57:06.186548 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186537    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:57:06.186548 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186547    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186551    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186555    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186558    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186561    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186564    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186567    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186570    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186574    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186579    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186582    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186586    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186589    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186592    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186595    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186598    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186601    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186604    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186607    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:57:06.186607 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186610    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186613    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186617    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186620    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186626    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186630    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186634    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186636    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186639    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186642    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186645    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186648    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186651    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186654    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186657    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186659    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186662    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186665    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186668    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:57:06.187132 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186670    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186673    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186676    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186679    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186681    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186684    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186687    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186690    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186693    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186697    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186700    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186702    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186705    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186708    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186711    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186715    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186718    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186721    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186724    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186727    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:57:06.187635 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186730    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186733    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186735    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186738    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186741    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186744    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186747    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186749    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186752    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186755    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186758    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186760    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186763    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186766    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186769    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186772    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186776    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186780    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186783    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186787    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:57:06.188243 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186790    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186794    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186797    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186801    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186805    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186808    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186811    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186815    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186818    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186821    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186823    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186828    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186831    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186834    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186836    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186840    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186843    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186845    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186848    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:57:06.188781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.186851    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.186857    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187040    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187046    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187050    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187053    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187056    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187058    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187061    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187064    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187066    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187069    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187072    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187075    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187078    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:57:06.189306 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187081    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187083    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187086    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187089    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187092    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187095    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187098    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187101    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187104    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187106    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187110    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187113    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187115    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187118    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187121    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187124    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187126    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187129    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187132    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187134    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:57:06.189720 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187137    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187140    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187143    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187146    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187149    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187151    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187154    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187157    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187159    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187162    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187165    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187168    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187171    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187173    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187176    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187180    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187184    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187187    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187190    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:57:06.190255 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187193    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187196    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187198    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187203    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187207    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187211    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187214    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187217    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187220    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187222    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187225    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187228    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187231    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187233    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187236    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187239    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187242    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187244    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187247    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:57:06.190749 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187250    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187252    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187255    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187258    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187261    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187264    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187267    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187269    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187272    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187274    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187277    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187280    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187283    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187286    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187289    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187293    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187296    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187299    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187303    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187306    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:57:06.191273 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187309    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187312    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187314    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187317    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187320    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187322    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187325    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:06.187327    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.187332    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.187487    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.189796    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.191327    2575 server.go:1019] "Starting client certificate rotation"
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.191441    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:57:06.191840 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.191487    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:57:06.193921 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.193906    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:57:06.195195 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.195176    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:57:06.203302 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.203283    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 28 17:57:06.205596 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.205578    2575 log.go:25] "Validated CRI v1 image API"
Jul 28 17:57:06.207411 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.207395    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 17:57:06.212080 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.212059    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 87ca15c9-5c37-4eda-b51a-665c442ca226:/dev/nvme0n1p4 c9e7ea3f-15e4-42d1-92c5-174643b83973:/dev/nvme0n1p3]
Jul 28 17:57:06.212138 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.212080    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 17:57:06.219574 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.219443    2575 manager.go:216] Machine: {Timestamp:2026-07-28 17:57:06.217655021 +0000 UTC m=+0.363207990 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099638 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25e0e287fd5efacd8d01abfc3c3869 SystemUUID:ec25e0e2-87fd-5efa-cd8d-01abfc3c3869 BootID:8742498c-7a5a-4466-9316-86faa38704ce Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3c:73:7f:b0:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3c:73:7f:b0:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:a3:eb:bd:8b:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 17:57:06.219574 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.219558    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 17:57:06.219722 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.219706    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 17:57:06.222405 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.222386    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:57:06.222815 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.222788    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 17:57:06.222985 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.222820    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-23.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 17:57:06.223571 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.223559    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 17:57:06.223611 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.223573    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 17:57:06.223611 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.223582    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:57:06.224249 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.224237    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:57:06.226007 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.225994    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 17:57:06.229402 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.229388    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:57:06.229530 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.229519    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 17:57:06.229574 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.229552    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 17:57:06.229574 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.229562    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 17:57:06.229633 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.229581    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 17:57:06.229633 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.229595    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 28 17:57:06.229633 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.229607    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 17:57:06.230063 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.230049    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:57:06.230109 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.230071    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:57:06.232732 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.232715    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 17:57:06.234449 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.234435    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 17:57:06.234507 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.234468    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 17:57:06.235420 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235408    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235425    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235432    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235437    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235446    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235452    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235461    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235466    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235473    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 17:57:06.235478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235478    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 17:57:06.235716 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235487    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 17:57:06.235716 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.235494    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 17:57:06.236200 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.236186    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 17:57:06.236200 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.236201    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 17:57:06.239697 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.239679    2575 server.go:1268] "Started kubelet"
Jul 28 17:57:06.239804 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.239753    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-23.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 17:57:06.239804 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.239777    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 17:57:06.239922 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.239841    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 17:57:06.239922 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.239812    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 17:57:06.240015 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.239933    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 17:57:06.240015 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.239970    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 17:57:06.240695 ip-10-0-132-23 systemd[1]: Started Kubernetes Kubelet.
Jul 28 17:57:06.241342 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.241219    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 17:57:06.241688 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.241670    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 17:57:06.245497 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.245478    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 17:57:06.248821 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.247855    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-23.ec2.internal.18c685cef638ce9b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-23.ec2.internal,UID:ip-10-0-132-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-23.ec2.internal,},FirstTimestamp:2026-07-28 17:57:06.239655579 +0000 UTC m=+0.385208547,LastTimestamp:2026-07-28 17:57:06.239655579 +0000 UTC m=+0.385208547,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-23.ec2.internal,}"
Jul 28 17:57:06.249070 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.249052    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cwlnz"
Jul 28 17:57:06.249813 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.249791    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 17:57:06.249813 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.249800    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 17:57:06.250011 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.249990    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.250412 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.250392    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 17:57:06.250412 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.250398    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 17:57:06.250547 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.250424    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 17:57:06.250547 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.250503    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 17:57:06.250547 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.250516    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 17:57:06.251011 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.250993    2575 factory.go:55] Registering systemd factory
Jul 28 17:57:06.251011 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.251013    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 28 17:57:06.251301 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.251283    2575 factory.go:153] Registering CRI-O factory
Jul 28 17:57:06.251301 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.251303    2575 factory.go:223] Registration of the crio container factory successfully
Jul 28 17:57:06.251424 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.251348    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 17:57:06.251424 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.251369    2575 factory.go:103] Registering Raw factory
Jul 28 17:57:06.251424 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.251380    2575 manager.go:1196] Started watching for new ooms in manager
Jul 28 17:57:06.251907 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.251890    2575 manager.go:349] Starting recovery of all containers
Jul 28 17:57:06.255328 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.255297    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 17:57:06.255576 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.255542    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 17:57:06.257913 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.257887    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cwlnz"
Jul 28 17:57:06.266251 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.266233    2575 manager.go:354] Recovery completed
Jul 28 17:57:06.271014 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.270998    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:57:06.272989 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.272964    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:57:06.273049 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.273007    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:57:06.273049 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.273023    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:57:06.273709 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.273695    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 17:57:06.273758 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.273709    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 17:57:06.273758 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.273726    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:57:06.273818 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.273770    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:57:06.274704 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.274686    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:57:06.274812 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.274720    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:57:06.274812 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.274735    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:57:06.275205 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.275187    2575 policy_none.go:49] "None policy: Start"
Jul 28 17:57:06.275205 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.275210    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 17:57:06.275205 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.275143    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-23.ec2.internal.18c685cef8357d31  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-23.ec2.internal,UID:ip-10-0-132-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-23.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-23.ec2.internal,},FirstTimestamp:2026-07-28 17:57:06.272992561 +0000 UTC m=+0.418545529,LastTimestamp:2026-07-28 17:57:06.272992561 +0000 UTC m=+0.418545529,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-23.ec2.internal,}"
Jul 28 17:57:06.275434 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.275228    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 17:57:06.276418 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.276399    2575 policy_none.go:47] "Start"
Jul 28 17:57:06.313948 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.313622    2575 manager.go:337] "Starting Device Plugin manager"
Jul 28 17:57:06.313948 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.313878    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 17:57:06.313948 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.313913    2575 server.go:87] "Starting device plugin registration server"
Jul 28 17:57:06.314291 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.314269    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 17:57:06.314354 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.314291    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 17:57:06.314401 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.314388    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 17:57:06.314500 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.314486    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 17:57:06.314563 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.314501    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 17:57:06.315098 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.315078    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 17:57:06.315186 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.315133    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.392310 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.392267    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 17:57:06.393598 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.393582    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 17:57:06.393680 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.393629    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 17:57:06.393680 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.393668    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 17:57:06.393788 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.393715    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 17:57:06.395614 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.395587    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:06.414787 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.414756    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:57:06.417090 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.417069    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:57:06.417179 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.417109    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:57:06.417179 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.417122    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:57:06.417179 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.417147    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.423346 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.423326    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.423412 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.423356    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-23.ec2.internal\": node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.437433 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.437408    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.494606 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.494564    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"]
Jul 28 17:57:06.494738 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.494649    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:57:06.495663 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.495645    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:57:06.495738 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.495677    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:57:06.495738 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.495698    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:57:06.497133 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497119    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:57:06.497268 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.497311 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497285    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:57:06.497930 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497915    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:57:06.498021 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497931    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:57:06.498021 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497937    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:57:06.498021 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497976    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:57:06.498021 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.497953    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:57:06.498217 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.498040    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:57:06.499405 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.499388    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.499496 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.499421    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:57:06.500247 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.500230    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:57:06.500326 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.500260    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:57:06.500326 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.500271    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:57:06.522703 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.522678    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-23.ec2.internal\" not found" node="ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.529236 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.529211    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-23.ec2.internal\" not found" node="ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.538256 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.538220    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.552075 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.552039    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/747991d5f1507a8a1f794fb236e2e067-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"747991d5f1507a8a1f794fb236e2e067\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.552075 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.552074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/747991d5f1507a8a1f794fb236e2e067-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"747991d5f1507a8a1f794fb236e2e067\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.552281 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.552095    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75df323d7741942773952d1b2e2ae384-config\") pod \"kube-apiserver-proxy-ip-10-0-132-23.ec2.internal\" (UID: \"75df323d7741942773952d1b2e2ae384\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.638463 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.638367    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.652541 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.652510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/747991d5f1507a8a1f794fb236e2e067-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"747991d5f1507a8a1f794fb236e2e067\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.652541 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.652541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/747991d5f1507a8a1f794fb236e2e067-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"747991d5f1507a8a1f794fb236e2e067\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.652739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.652564    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75df323d7741942773952d1b2e2ae384-config\") pod \"kube-apiserver-proxy-ip-10-0-132-23.ec2.internal\" (UID: \"75df323d7741942773952d1b2e2ae384\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.652739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.652617    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/747991d5f1507a8a1f794fb236e2e067-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"747991d5f1507a8a1f794fb236e2e067\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.652739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.652629    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/747991d5f1507a8a1f794fb236e2e067-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"747991d5f1507a8a1f794fb236e2e067\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.652739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.652660    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75df323d7741942773952d1b2e2ae384-config\") pod \"kube-apiserver-proxy-ip-10-0-132-23.ec2.internal\" (UID: \"75df323d7741942773952d1b2e2ae384\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.738578 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.738538    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.830727 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.830680    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.833012 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:06.832989    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:06.839695 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.839670    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:06.941923 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:06.941794    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:07.042560 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:07.042520    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:07.143243 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:07.143189    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:07.193083 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.193005    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 17:57:07.193539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.193147    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 17:57:07.243826 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:07.243791    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:07.263249 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.263204    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:52:07 +0000 UTC" deadline="2028-02-23 06:24:58.213010491 +0000 UTC"
Jul 28 17:57:07.263249 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.263245    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13788h27m50.949769259s"
Jul 28 17:57:07.344977 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:07.344941    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:07.359599 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.359572    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:07.445780 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:07.445703    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:07.460195 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:07.460155    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod747991d5f1507a8a1f794fb236e2e067.slice/crio-a646a192cbb9b3c8305c649cfea7a520edc79d22123855a61383998c9ca68ed7 WatchSource:0}: Error finding container a646a192cbb9b3c8305c649cfea7a520edc79d22123855a61383998c9ca68ed7: Status 404 returned error can't find the container with id a646a192cbb9b3c8305c649cfea7a520edc79d22123855a61383998c9ca68ed7
Jul 28 17:57:07.460796 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:07.460767    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75df323d7741942773952d1b2e2ae384.slice/crio-8a44de2e0d3dbcd23731a3dd332f7775ec46d62bdb6c103d426a34a7c2476af9 WatchSource:0}: Error finding container 8a44de2e0d3dbcd23731a3dd332f7775ec46d62bdb6c103d426a34a7c2476af9: Status 404 returned error can't find the container with id 8a44de2e0d3dbcd23731a3dd332f7775ec46d62bdb6c103d426a34a7c2476af9
Jul 28 17:57:07.465014 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.464994    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:57:07.470311 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.470292    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:07.550666 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:07.550622    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found"
Jul 28 17:57:07.559198 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.559170    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:07.651306 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.651275    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:07.665667 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.665642    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:57:07.666863 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.666846    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"
Jul 28 17:57:07.674205 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:07.674185    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:57:08.230760 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.230721    2575 apiserver.go:52] "Watching apiserver"
Jul 28 17:57:08.239086 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.239050    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 17:57:08.239578 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.239531    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vdwk2","kube-system/konnectivity-agent-ltcl9","kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal","openshift-dns/node-resolver-zqss8","openshift-multus/multus-8whqv","openshift-multus/network-metrics-daemon-jck5m","openshift-network-diagnostics/network-check-target-q8q24","openshift-ovn-kubernetes/ovnkube-node-88wzn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd","openshift-cluster-node-tuning-operator/tuned-lsjq5","openshift-image-registry/node-ca-nvwm8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal","openshift-multus/multus-additional-cni-plugins-xs4kr","openshift-network-operator/iptables-alerter-rbrw2"]
Jul 28 17:57:08.241171 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.241145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:08.241284 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.241245    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:08.242503 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.242477    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.244056 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.244027    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.245488 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.245271    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 17:57:08.245488 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.245291    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fd8pz\""
Jul 28 17:57:08.245897 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.245869    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 17:57:08.246230 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.246209    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.246569 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.246332    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 17:57:08.246569 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.246433    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 17:57:08.246569 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.246333    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 17:57:08.249992 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.249701    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 17:57:08.250103 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.250077    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:08.250164 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.250136    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qpx4p\""
Jul 28 17:57:08.250293 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.250268    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:08.250621 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.250602    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 17:57:08.250738 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.250714    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.250867 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.250850    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 17:57:08.251140 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.251118    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mgxst\""
Jul 28 17:57:08.251813 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.251321    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 17:57:08.253770 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.253548    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:08.253770 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.253619    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:08.253770 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.253670    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 17:57:08.253770 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.253762    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fd2r7\""
Jul 28 17:57:08.255346 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.255326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.256537 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.256429    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.256537 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.256504    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:08.257947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.257922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.259558 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.259558 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259252    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 17:57:08.259717 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259572    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 17:57:08.259717 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259644    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rdxdr\""
Jul 28 17:57:08.259806 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259744    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 17:57:08.259855 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb2fl\" (UniqueName: \"kubernetes.io/projected/599dce4a-55e5-489e-bd51-5c4251e9c3ed-kube-api-access-nb2fl\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.259916 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259869    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-cnibin\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.259916 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-cni-bin\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260119 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.259979    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-cni-multus\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260119 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:08.260119 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-cnibin\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.260119 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-conf-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260119 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-cni-binary-copy\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260151    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/696d718f-48c0-48c8-aca4-a83421d1f85e-host-slash\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260180    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-netns\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260198    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbn8h\" (UniqueName: \"kubernetes.io/projected/696d718f-48c0-48c8-aca4-a83421d1f85e-kube-api-access-tbn8h\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260216    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d52be987-69c6-4133-a75b-6f5453ca3552-cni-binary-copy\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260232    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d52be987-69c6-4133-a75b-6f5453ca3552-multus-daemon-config\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-etc-kubernetes\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rv6k\" (UniqueName: \"kubernetes.io/projected/d52be987-69c6-4133-a75b-6f5453ca3552-kube-api-access-6rv6k\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260298    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjxgt\" (UniqueName: \"kubernetes.io/projected/bf5686ec-91c6-42a7-8436-1df336b0221c-kube-api-access-kjxgt\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260314    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-system-cni-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.260432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260426    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-socket-dir-parent\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260442    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-hostroot\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260462    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qcstf\""
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260469    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-multus-certs\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260485    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/696d718f-48c0-48c8-aca4-a83421d1f85e-iptables-alerter-script\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260500    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-os-release\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-k8s-cni-cncf-io\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260544    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzl2l\" (UniqueName: \"kubernetes.io/projected/5f6924da-d133-4898-8aa4-0a1aa892ff2e-kube-api-access-nzl2l\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260576    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260627    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599dce4a-55e5-489e-bd51-5c4251e9c3ed-tmp-dir\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-cni-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260655    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-os-release\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260678    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-kubelet\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260711    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-system-cni-dir\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.261227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.260729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/599dce4a-55e5-489e-bd51-5c4251e9c3ed-hosts-file\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.262085 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.261799    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.263454 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.263428    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.265259 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.265239    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pjqx7\""
Jul 28 17:57:08.265347 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.265278    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:08.265627 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.265602    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:08.266089 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.266068    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 17:57:08.266228 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.266127    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 17:57:08.266294 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.266145    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 17:57:08.266372 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.266171    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 17:57:08.266634 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.266618    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-99zv4\""
Jul 28 17:57:08.267705 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.267673    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xtwq9\""
Jul 28 17:57:08.269242 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.268786    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 17:57:08.269242 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.268918    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 17:57:08.269242 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.269046    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 17:57:08.269242 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.269097    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 17:57:08.270064 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.270046    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 17:57:08.273284 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.273261    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:57:08.305679 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.305645    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9qsls"
Jul 28 17:57:08.314450 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.314413    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9qsls"
Jul 28 17:57:08.352763 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.352715    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 17:57:08.361218 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:08.361218 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361221    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-os-release\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361241    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-k8s-cni-cncf-io\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361270    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55e27a0d-949a-44da-9368-a62947420498-dbus\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-kubelet\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysconfig\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361352    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-systemd\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-socket-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361434    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-os-release\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.361464 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-kubelet\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-system-cni-dir\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361513    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb2fl\" (UniqueName: \"kubernetes.io/projected/599dce4a-55e5-489e-bd51-5c4251e9c3ed-kube-api-access-nb2fl\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-cnibin\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55e27a0d-949a-44da-9368-a62947420498-kubelet-config\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ls4\" (UniqueName: \"kubernetes.io/projected/84007fd6-0b1b-45cb-b78c-01cff9dca00d-kube-api-access-57ls4\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361605    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-modprobe-d\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361626    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4smh\" (UniqueName: \"kubernetes.io/projected/e1a7dab8-512b-4b50-bad4-6978e3e99045-kube-api-access-v4smh\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361650    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-cnibin\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-conf-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-var-lib-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361697    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovnkube-config\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361714    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-kubernetes\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361738    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-tuned\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361765    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-cni-binary-copy\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361790    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/696d718f-48c0-48c8-aca4-a83421d1f85e-host-slash\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-systemd-units\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.361947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361829    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-systemd\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361844    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-run\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbn8h\" (UniqueName: \"kubernetes.io/projected/696d718f-48c0-48c8-aca4-a83421d1f85e-kube-api-access-tbn8h\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-etc-kubernetes\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rv6k\" (UniqueName: \"kubernetes.io/projected/d52be987-69c6-4133-a75b-6f5453ca3552-kube-api-access-6rv6k\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c27ba9c5-86c1-4f79-a3d7-425d5c2d750f-konnectivity-ca\") pod \"konnectivity-agent-ltcl9\" (UID: \"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f\") " pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.361968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-etc-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-node-log\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362024    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr74k\" (UniqueName: \"kubernetes.io/projected/3280d6cf-afd3-4d04-b0e7-72db473e4867-kube-api-access-pr74k\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjxgt\" (UniqueName: \"kubernetes.io/projected/bf5686ec-91c6-42a7-8436-1df336b0221c-kube-api-access-kjxgt\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362059    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-system-cni-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-lib-modules\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/351f44fb-f7d8-4c17-8128-c823d57eeb92-tmp\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-registration-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362159    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/696d718f-48c0-48c8-aca4-a83421d1f85e-iptables-alerter-script\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362183    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-ovn\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysctl-d\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.362809 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzl2l\" (UniqueName: \"kubernetes.io/projected/5f6924da-d133-4898-8aa4-0a1aa892ff2e-kube-api-access-nzl2l\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-run-netns\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-cni-bin\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysctl-conf\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362293    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362310    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599dce4a-55e5-489e-bd51-5c4251e9c3ed-tmp-dir\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-cni-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-device-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362378    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/599dce4a-55e5-489e-bd51-5c4251e9c3ed-hosts-file\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-cni-bin\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362434    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-cni-multus\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362459    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362612    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-cni-bin\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362805    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-cni-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362871    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-system-cni-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.362992    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/599dce4a-55e5-489e-bd51-5c4251e9c3ed-hosts-file\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363037    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/696d718f-48c0-48c8-aca4-a83421d1f85e-host-slash\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.363539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-cni-multus\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363154    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-var-lib-kubelet\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-os-release\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363221    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-system-cni-dir\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363227    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-cni-binary-copy\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363295    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-os-release\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363298    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-k8s-cni-cncf-io\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.363314    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363346    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-cnibin\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-conf-dir\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.363388    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs podName:5f6924da-d133-4898-8aa4-0a1aa892ff2e nodeName:}" failed. No retries permitted until 2026-07-28 17:57:08.863364369 +0000 UTC m=+3.008917336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs") pod "network-metrics-daemon-jck5m" (UID: "5f6924da-d133-4898-8aa4-0a1aa892ff2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-run-ovn-kubernetes\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363308    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-etc-kubernetes\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599dce4a-55e5-489e-bd51-5c4251e9c3ed-tmp-dir\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363502    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-cnibin\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363525    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-var-lib-kubelet\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.364453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363580    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1a7dab8-512b-4b50-bad4-6978e3e99045-host\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363630    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1a7dab8-512b-4b50-bad4-6978e3e99045-serviceca\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-log-socket\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-env-overrides\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363879    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf5686ec-91c6-42a7-8436-1df336b0221c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c27ba9c5-86c1-4f79-a3d7-425d5c2d750f-agent-certs\") pod \"konnectivity-agent-ltcl9\" (UID: \"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f\") " pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363932    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovn-node-metrics-cert\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.363988    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/696d718f-48c0-48c8-aca4-a83421d1f85e-iptables-alerter-script\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf5686ec-91c6-42a7-8436-1df336b0221c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-netns\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364087    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d52be987-69c6-4133-a75b-6f5453ca3552-cni-binary-copy\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d52be987-69c6-4133-a75b-6f5453ca3552-multus-daemon-config\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364145    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.365143 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364182    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-slash\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovnkube-script-lib\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-socket-dir-parent\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-hostroot\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-multus-certs\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364352    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-cni-netd\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-host\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364405    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-etc-selinux\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364430    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-sys\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364486    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzpsm\" (UniqueName: \"kubernetes.io/projected/351f44fb-f7d8-4c17-8128-c823d57eeb92-kube-api-access-lzpsm\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364542    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-sys-fs\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d52be987-69c6-4133-a75b-6f5453ca3552-multus-daemon-config\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-multus-socket-dir-parent\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-hostroot\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-multus-certs\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.364974    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d52be987-69c6-4133-a75b-6f5453ca3552-host-run-netns\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.365974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.365054    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d52be987-69c6-4133-a75b-6f5453ca3552-cni-binary-copy\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.372294 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.372242    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 17:57:08.376204 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.376173    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:08.376204 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.376203    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:08.376448 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.376220    2575 projected.go:196] Error preparing data for projected volume kube-api-access-gbwr8 for pod openshift-network-diagnostics/network-check-target-q8q24: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:08.376448 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.376310    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8 podName:724f5f04-2dc2-4aca-8088-812f03ebf9f8 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:08.876288372 +0000 UTC m=+3.021841347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gbwr8" (UniqueName: "kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8") pod "network-check-target-q8q24" (UID: "724f5f04-2dc2-4aca-8088-812f03ebf9f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:08.376549 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.376453    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzl2l\" (UniqueName: \"kubernetes.io/projected/5f6924da-d133-4898-8aa4-0a1aa892ff2e-kube-api-access-nzl2l\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:08.378591 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.378559    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb2fl\" (UniqueName: \"kubernetes.io/projected/599dce4a-55e5-489e-bd51-5c4251e9c3ed-kube-api-access-nb2fl\") pod \"node-resolver-zqss8\" (UID: \"599dce4a-55e5-489e-bd51-5c4251e9c3ed\") " pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.379003 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.378983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjxgt\" (UniqueName: \"kubernetes.io/projected/bf5686ec-91c6-42a7-8436-1df336b0221c-kube-api-access-kjxgt\") pod \"multus-additional-cni-plugins-xs4kr\" (UID: \"bf5686ec-91c6-42a7-8436-1df336b0221c\") " pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.379334 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.379309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rv6k\" (UniqueName: \"kubernetes.io/projected/d52be987-69c6-4133-a75b-6f5453ca3552-kube-api-access-6rv6k\") pod \"multus-8whqv\" (UID: \"d52be987-69c6-4133-a75b-6f5453ca3552\") " pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.380229 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.380173    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbn8h\" (UniqueName: \"kubernetes.io/projected/696d718f-48c0-48c8-aca4-a83421d1f85e-kube-api-access-tbn8h\") pod \"iptables-alerter-rbrw2\" (UID: \"696d718f-48c0-48c8-aca4-a83421d1f85e\") " pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.399672 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.399607    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" event={"ID":"747991d5f1507a8a1f794fb236e2e067","Type":"ContainerStarted","Data":"a646a192cbb9b3c8305c649cfea7a520edc79d22123855a61383998c9ca68ed7"}
Jul 28 17:57:08.402284 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.402191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" event={"ID":"75df323d7741942773952d1b2e2ae384","Type":"ContainerStarted","Data":"8a44de2e0d3dbcd23731a3dd332f7775ec46d62bdb6c103d426a34a7c2476af9"}
Jul 28 17:57:08.465166 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-run-ovn-kubernetes\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465339 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-var-lib-kubelet\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.465339 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465202    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1a7dab8-512b-4b50-bad4-6978e3e99045-host\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.465339 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465256    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1a7dab8-512b-4b50-bad4-6978e3e99045-host\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.465339 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-run-ovn-kubernetes\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465339 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465290    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-var-lib-kubelet\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.465339 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465302    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1a7dab8-512b-4b50-bad4-6978e3e99045-serviceca\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.465339 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-log-socket\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465390    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-log-socket\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465422    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-env-overrides\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c27ba9c5-86c1-4f79-a3d7-425d5c2d750f-agent-certs\") pod \"konnectivity-agent-ltcl9\" (UID: \"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f\") " pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovn-node-metrics-cert\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465535    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465583    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465609    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-slash\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465634    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovnkube-script-lib\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-cni-netd\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.465683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-host\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.465712    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.465776    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret podName:55e27a0d-949a-44da-9368-a62947420498 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:08.965755525 +0000 UTC m=+3.111308491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret") pod "global-pull-secret-syncer-vdwk2" (UID: "55e27a0d-949a-44da-9368-a62947420498") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1a7dab8-512b-4b50-bad4-6978e3e99045-serviceca\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465825    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-etc-selinux\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-etc-selinux\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465863    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-sys\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-slash\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465944    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzpsm\" (UniqueName: \"kubernetes.io/projected/351f44fb-f7d8-4c17-8128-c823d57eeb92-kube-api-access-lzpsm\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.465976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-sys-fs\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466044    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55e27a0d-949a-44da-9368-a62947420498-dbus\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466076    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-kubelet\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466068    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-env-overrides\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466103    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysconfig\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466131    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-systemd\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466157    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-socket-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.466262 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466165    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466191    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55e27a0d-949a-44da-9368-a62947420498-kubelet-config\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466217    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ls4\" (UniqueName: \"kubernetes.io/projected/84007fd6-0b1b-45cb-b78c-01cff9dca00d-kube-api-access-57ls4\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466224    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysconfig\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-modprobe-d\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4smh\" (UniqueName: \"kubernetes.io/projected/e1a7dab8-512b-4b50-bad4-6978e3e99045-kube-api-access-v4smh\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466276    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-sys-fs\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-var-lib-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466318    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-kubelet\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovnkube-config\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-kubernetes\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-host\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-tuned\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466401    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovnkube-script-lib\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-systemd-units\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466430    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-systemd\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55e27a0d-949a-44da-9368-a62947420498-kubelet-config\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.467093 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-systemd\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466375    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-cni-netd\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-var-lib-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466482    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-run\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-socket-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-modprobe-d\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466683    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-kubernetes\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466716    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c27ba9c5-86c1-4f79-a3d7-425d5c2d750f-konnectivity-ca\") pod \"konnectivity-agent-ltcl9\" (UID: \"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f\") " pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466740    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55e27a0d-949a-44da-9368-a62947420498-dbus\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-etc-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466833    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-run\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466874    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-sys\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466906    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-systemd\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovnkube-config\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-systemd-units\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.466976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-node-log\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr74k\" (UniqueName: \"kubernetes.io/projected/3280d6cf-afd3-4d04-b0e7-72db473e4867-kube-api-access-pr74k\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467035    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-lib-modules\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.467880 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/351f44fb-f7d8-4c17-8128-c823d57eeb92-tmp\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467095    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-etc-openvswitch\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-registration-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467207    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-lib-modules\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467281    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-node-log\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-ovn\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-registration-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysctl-d\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467462    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-run-ovn\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467564    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysctl-d\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467580    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c27ba9c5-86c1-4f79-a3d7-425d5c2d750f-konnectivity-ca\") pod \"konnectivity-agent-ltcl9\" (UID: \"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f\") " pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467606    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-run-netns\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-cni-bin\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467670    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-run-netns\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84007fd6-0b1b-45cb-b78c-01cff9dca00d-host-cni-bin\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysctl-conf\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467853    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-device-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.468869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467909    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-sysctl-conf\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.469770 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.467920    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3280d6cf-afd3-4d04-b0e7-72db473e4867-device-dir\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.470096 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.470076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/351f44fb-f7d8-4c17-8128-c823d57eeb92-tmp\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.470851 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.470822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/351f44fb-f7d8-4c17-8128-c823d57eeb92-etc-tuned\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.470966 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.470852    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c27ba9c5-86c1-4f79-a3d7-425d5c2d750f-agent-certs\") pod \"konnectivity-agent-ltcl9\" (UID: \"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f\") " pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.472516 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.472491    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84007fd6-0b1b-45cb-b78c-01cff9dca00d-ovn-node-metrics-cert\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.478000 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.477967    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4smh\" (UniqueName: \"kubernetes.io/projected/e1a7dab8-512b-4b50-bad4-6978e3e99045-kube-api-access-v4smh\") pod \"node-ca-nvwm8\" (UID: \"e1a7dab8-512b-4b50-bad4-6978e3e99045\") " pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.478280 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.478257    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr74k\" (UniqueName: \"kubernetes.io/projected/3280d6cf-afd3-4d04-b0e7-72db473e4867-kube-api-access-pr74k\") pod \"aws-ebs-csi-driver-node-qc2rd\" (UID: \"3280d6cf-afd3-4d04-b0e7-72db473e4867\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.478416 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.478377    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzpsm\" (UniqueName: \"kubernetes.io/projected/351f44fb-f7d8-4c17-8128-c823d57eeb92-kube-api-access-lzpsm\") pod \"tuned-lsjq5\" (UID: \"351f44fb-f7d8-4c17-8128-c823d57eeb92\") " pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.478741 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.478695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ls4\" (UniqueName: \"kubernetes.io/projected/84007fd6-0b1b-45cb-b78c-01cff9dca00d-kube-api-access-57ls4\") pod \"ovnkube-node-88wzn\" (UID: \"84007fd6-0b1b-45cb-b78c-01cff9dca00d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.560507 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.560411    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xs4kr"
Jul 28 17:57:08.571042 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.571004    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rbrw2"
Jul 28 17:57:08.582353 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.582319    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqss8"
Jul 28 17:57:08.584646 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.584604    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:08.588496 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.588471    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8whqv"
Jul 28 17:57:08.597846 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.597809    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd"
Jul 28 17:57:08.604164 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.604130    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:08.615099 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.615068    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lsjq5"
Jul 28 17:57:08.624197 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.624164    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvwm8"
Jul 28 17:57:08.631806 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.631579    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:08.871264 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.871162    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:08.871430 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.871348    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:08.871430 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.871425    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs podName:5f6924da-d133-4898-8aa4-0a1aa892ff2e nodeName:}" failed. No retries permitted until 2026-07-28 17:57:09.87140896 +0000 UTC m=+4.016961913 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs") pod "network-metrics-daemon-jck5m" (UID: "5f6924da-d133-4898-8aa4-0a1aa892ff2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:08.971969 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.971931    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:08.971969 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:08.971977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:08.972231 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.972131    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:08.972231 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.972198    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:08.972231 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.972212    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret podName:55e27a0d-949a-44da-9368-a62947420498 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:09.972189556 +0000 UTC m=+4.117742520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret") pod "global-pull-secret-syncer-vdwk2" (UID: "55e27a0d-949a-44da-9368-a62947420498") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:08.972231 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.972217    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:08.972231 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.972233    2575 projected.go:196] Error preparing data for projected volume kube-api-access-gbwr8 for pod openshift-network-diagnostics/network-check-target-q8q24: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:08.972518 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:08.972274    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8 podName:724f5f04-2dc2-4aca-8088-812f03ebf9f8 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:09.972262403 +0000 UTC m=+4.117815363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gbwr8" (UniqueName: "kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8") pod "network-check-target-q8q24" (UID: "724f5f04-2dc2-4aca-8088-812f03ebf9f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:09.317578 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:09.317439    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:52:09 +0000 UTC" deadline="2028-05-06 20:50:00.848912578 +0000 UTC"
Jul 28 17:57:09.317578 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:09.317505    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15554h52m51.531411039s"
Jul 28 17:57:09.394719 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:09.394677    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:09.394944 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.394813    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:09.508374 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:09.508336    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84007fd6_0b1b_45cb_b78c_01cff9dca00d.slice/crio-3847a9c28da7061386fb5aec9115ff19166aec1c96fb569f8c72090a6c93a4eb WatchSource:0}: Error finding container 3847a9c28da7061386fb5aec9115ff19166aec1c96fb569f8c72090a6c93a4eb: Status 404 returned error can't find the container with id 3847a9c28da7061386fb5aec9115ff19166aec1c96fb569f8c72090a6c93a4eb
Jul 28 17:57:09.515648 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:09.514948    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52be987_69c6_4133_a75b_6f5453ca3552.slice/crio-f7b24571b58b75ce31355b3fc098256dafaff1b0441b086ad357a59c87b17543 WatchSource:0}: Error finding container f7b24571b58b75ce31355b3fc098256dafaff1b0441b086ad357a59c87b17543: Status 404 returned error can't find the container with id f7b24571b58b75ce31355b3fc098256dafaff1b0441b086ad357a59c87b17543
Jul 28 17:57:09.516112 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:09.516085    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod599dce4a_55e5_489e_bd51_5c4251e9c3ed.slice/crio-5f39035fef089d9f2dc3540883e30843738957af22d8f9f68e6b38c7e84d20c9 WatchSource:0}: Error finding container 5f39035fef089d9f2dc3540883e30843738957af22d8f9f68e6b38c7e84d20c9: Status 404 returned error can't find the container with id 5f39035fef089d9f2dc3540883e30843738957af22d8f9f68e6b38c7e84d20c9
Jul 28 17:57:09.880545 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:09.880421    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:09.880728 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.880587    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:09.880728 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.880667    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs podName:5f6924da-d133-4898-8aa4-0a1aa892ff2e nodeName:}" failed. No retries permitted until 2026-07-28 17:57:11.880650764 +0000 UTC m=+6.026203723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs") pod "network-metrics-daemon-jck5m" (UID: "5f6924da-d133-4898-8aa4-0a1aa892ff2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:09.981318 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:09.981276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:09.981507 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:09.981344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:09.981583 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.981527    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:09.981583 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.981549    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:09.981583 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.981563    2575 projected.go:196] Error preparing data for projected volume kube-api-access-gbwr8 for pod openshift-network-diagnostics/network-check-target-q8q24: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:09.981748 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.981627    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8 podName:724f5f04-2dc2-4aca-8088-812f03ebf9f8 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:11.981608499 +0000 UTC m=+6.127161467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gbwr8" (UniqueName: "kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8") pod "network-check-target-q8q24" (UID: "724f5f04-2dc2-4aca-8088-812f03ebf9f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:09.982072 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.982051    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:09.982166 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:09.982109    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret podName:55e27a0d-949a-44da-9368-a62947420498 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:11.982093847 +0000 UTC m=+6.127646810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret") pod "global-pull-secret-syncer-vdwk2" (UID: "55e27a0d-949a-44da-9368-a62947420498") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:10.318192 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.318110    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:52:09 +0000 UTC" deadline="2028-03-05 15:00:03.367788671 +0000 UTC"
Jul 28 17:57:10.318192 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.318152    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14061h2m53.049642318s"
Jul 28 17:57:10.395324 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.395081    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:10.395324 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.395086    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:10.395324 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:10.395273    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:10.395324 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:10.395318    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:10.427152 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.427109    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lsjq5" event={"ID":"351f44fb-f7d8-4c17-8128-c823d57eeb92","Type":"ContainerStarted","Data":"f6b1c863ff3901168616595020cadc46423c047c5132c84636af1734b3887973"}
Jul 28 17:57:10.443343 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.443257    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerStarted","Data":"6c1aeb45c46bfd0f2887282cf19be52d22e9383fdacd613ebe9712e90a45a407"}
Jul 28 17:57:10.452666 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.452567    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ltcl9" event={"ID":"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f","Type":"ContainerStarted","Data":"182b25ddb39c0cb568074bfcb1d79f3f4acebe9c41970617039e34d1c1e4222d"}
Jul 28 17:57:10.456594 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.456505    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rbrw2" event={"ID":"696d718f-48c0-48c8-aca4-a83421d1f85e","Type":"ContainerStarted","Data":"e39e463b48201ceb957a962fa481bdefaab10d3ee9fb99b1089eaf9720a61556"}
Jul 28 17:57:10.458680 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.458593    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd" event={"ID":"3280d6cf-afd3-4d04-b0e7-72db473e4867","Type":"ContainerStarted","Data":"488767d98c7cc81b3b8ca307371cc373317fcf9e35dcd617d952e50a1a0f1f48"}
Jul 28 17:57:10.466779 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.466603    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvwm8" event={"ID":"e1a7dab8-512b-4b50-bad4-6978e3e99045","Type":"ContainerStarted","Data":"2c6a636316b663839603558a088497d5e283f865c377b3fdad54cc81c40d9bee"}
Jul 28 17:57:10.485624 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.485531    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqss8" event={"ID":"599dce4a-55e5-489e-bd51-5c4251e9c3ed","Type":"ContainerStarted","Data":"5f39035fef089d9f2dc3540883e30843738957af22d8f9f68e6b38c7e84d20c9"}
Jul 28 17:57:10.500565 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.500418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8whqv" event={"ID":"d52be987-69c6-4133-a75b-6f5453ca3552","Type":"ContainerStarted","Data":"f7b24571b58b75ce31355b3fc098256dafaff1b0441b086ad357a59c87b17543"}
Jul 28 17:57:10.506336 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.506204    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"3847a9c28da7061386fb5aec9115ff19166aec1c96fb569f8c72090a6c93a4eb"}
Jul 28 17:57:10.530201 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.530157    2575 generic.go:356] "Generic (PLEG): container finished" podID="747991d5f1507a8a1f794fb236e2e067" containerID="535a530288cf4cf93080651902e87faf936994fa4ef0af3ac478ef6d4a699737" exitCode=0
Jul 28 17:57:10.530426 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.530285    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" event={"ID":"747991d5f1507a8a1f794fb236e2e067","Type":"ContainerDied","Data":"535a530288cf4cf93080651902e87faf936994fa4ef0af3ac478ef6d4a699737"}
Jul 28 17:57:10.558885 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.558838    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" event={"ID":"75df323d7741942773952d1b2e2ae384","Type":"ContainerStarted","Data":"1fd0562db8994628bbe9644994574158d31f540bf45430df36e9dabf52c5ccac"}
Jul 28 17:57:10.577357 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:10.576958    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" podStartSLOduration=2.5769153879999998 podStartE2EDuration="2.576915388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:10.575988724 +0000 UTC m=+4.721541704" watchObservedRunningTime="2026-07-28 17:57:10.576915388 +0000 UTC m=+4.722468367"
Jul 28 17:57:11.394113 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:11.394077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:11.394650 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.394233    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:11.599539 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:11.599471    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" event={"ID":"747991d5f1507a8a1f794fb236e2e067","Type":"ContainerStarted","Data":"f32e5092fab0c25e265c6c97177f1b53b068269dcb265160b597c92598b07a68"}
Jul 28 17:57:11.898471 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:11.898423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:11.898655 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.898623    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:11.898761 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.898690    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs podName:5f6924da-d133-4898-8aa4-0a1aa892ff2e nodeName:}" failed. No retries permitted until 2026-07-28 17:57:15.898672532 +0000 UTC m=+10.044225494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs") pod "network-metrics-daemon-jck5m" (UID: "5f6924da-d133-4898-8aa4-0a1aa892ff2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:11.999000 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:11.998952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:11.999201 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:11.999017    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:11.999201 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.999197    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:11.999316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.999217    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:11.999316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.999231    2575 projected.go:196] Error preparing data for projected volume kube-api-access-gbwr8 for pod openshift-network-diagnostics/network-check-target-q8q24: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:11.999316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.999295    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8 podName:724f5f04-2dc2-4aca-8088-812f03ebf9f8 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:15.999274296 +0000 UTC m=+10.144827251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gbwr8" (UniqueName: "kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8") pod "network-check-target-q8q24" (UID: "724f5f04-2dc2-4aca-8088-812f03ebf9f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:11.999758 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.999729    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:11.999859 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:11.999794    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret podName:55e27a0d-949a-44da-9368-a62947420498 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:15.999777787 +0000 UTC m=+10.145330746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret") pod "global-pull-secret-syncer-vdwk2" (UID: "55e27a0d-949a-44da-9368-a62947420498") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:12.395246 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:12.395086    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:12.395728 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:12.395250    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:12.395728 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:12.395085    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:12.395728 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:12.395700    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:13.394850 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:13.394314    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:13.394850 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:13.394461    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:14.394583 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:14.394549    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:14.395082 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:14.394601    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:14.395082 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:14.394724    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:14.395082 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:14.394967    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:15.394027 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:15.393944    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:15.394290 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:15.394092    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:15.935018 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:15.934972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:15.935555 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:15.935162    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:15.935555 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:15.935249    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs podName:5f6924da-d133-4898-8aa4-0a1aa892ff2e nodeName:}" failed. No retries permitted until 2026-07-28 17:57:23.935226367 +0000 UTC m=+18.080779344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs") pod "network-metrics-daemon-jck5m" (UID: "5f6924da-d133-4898-8aa4-0a1aa892ff2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:16.035760    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:16.035834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.035942    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.035945    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.035968    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.035978    2575 projected.go:196] Error preparing data for projected volume kube-api-access-gbwr8 for pod openshift-network-diagnostics/network-check-target-q8q24: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.035993    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret podName:55e27a0d-949a-44da-9368-a62947420498 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:24.035976901 +0000 UTC m=+18.181529860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret") pod "global-pull-secret-syncer-vdwk2" (UID: "55e27a0d-949a-44da-9368-a62947420498") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:16.036316 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.036023    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8 podName:724f5f04-2dc2-4aca-8088-812f03ebf9f8 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:24.036009395 +0000 UTC m=+18.181562348 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gbwr8" (UniqueName: "kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8") pod "network-check-target-q8q24" (UID: "724f5f04-2dc2-4aca-8088-812f03ebf9f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:16.395362 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:16.395273    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:16.395532 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.395393    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:16.395821 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:16.395799    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:16.395954 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:16.395915    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:17.394700 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:17.394666    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:17.395266 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:17.394784    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:18.395565 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:18.394971    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:18.395565 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:18.395110    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:18.395565 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:18.394972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:18.395565 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:18.395523    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:19.394308 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:19.394270    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:19.394487 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:19.394395    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:20.397647 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:20.397621    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:20.398096 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:20.397625    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:20.398096 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:20.397718    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:20.398096 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:20.397820    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:21.394536 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:21.394356    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:21.394536 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:21.394492    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:22.394336 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:22.394299    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:22.394769 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:22.394448    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:22.394769 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:22.394478    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:22.394769 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:22.394599    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:23.394704 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:23.394658    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:23.395290 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:23.394788    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:23.997768 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:23.997727    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:23.997965 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:23.997864    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:23.997965 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:23.997921    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs podName:5f6924da-d133-4898-8aa4-0a1aa892ff2e nodeName:}" failed. No retries permitted until 2026-07-28 17:57:39.997906096 +0000 UTC m=+34.143459052 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs") pod "network-metrics-daemon-jck5m" (UID: "5f6924da-d133-4898-8aa4-0a1aa892ff2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:24.098832 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:24.098632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:24.098832 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:24.098677    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:24.098832 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.098783    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:24.098832 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.098798    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:24.098832 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.098814    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:24.098832 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.098826    2575 projected.go:196] Error preparing data for projected volume kube-api-access-gbwr8 for pod openshift-network-diagnostics/network-check-target-q8q24: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:24.099205 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.098869    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret podName:55e27a0d-949a-44da-9368-a62947420498 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:40.098835521 +0000 UTC m=+34.244388497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret") pod "global-pull-secret-syncer-vdwk2" (UID: "55e27a0d-949a-44da-9368-a62947420498") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:24.099205 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.098884    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8 podName:724f5f04-2dc2-4aca-8088-812f03ebf9f8 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:40.098877725 +0000 UTC m=+34.244430681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gbwr8" (UniqueName: "kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8") pod "network-check-target-q8q24" (UID: "724f5f04-2dc2-4aca-8088-812f03ebf9f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:24.394820 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:24.394733    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:24.395361 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:24.394748    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:24.395361 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.394901    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:24.395361 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:24.394950    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:25.394226 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:25.394194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:25.394412 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:25.394313    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:26.394721 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:26.394685    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:26.395208 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:26.394809    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:26.395208 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:26.394888    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:26.395208 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:26.394992    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:27.394206 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:27.394164    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:27.394378 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:27.394292    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:28.394824 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:28.394784    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:28.395288 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:28.394836    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:28.395288 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:28.394951    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:28.395288 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:28.395109    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:29.394173 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.393898    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:29.394300 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:29.394274    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:29.639375 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.639162    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lsjq5" event={"ID":"351f44fb-f7d8-4c17-8128-c823d57eeb92","Type":"ContainerStarted","Data":"78ebfe4275090ea4d1da77d73f614e6891ce28717d49bf3b343a409866d81cf0"}
Jul 28 17:57:29.640672 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.640646    2575 generic.go:356] "Generic (PLEG): container finished" podID="bf5686ec-91c6-42a7-8436-1df336b0221c" containerID="3082937b4bec5d3d876824a15e30b540e8bb2b9e6014bb59ba8a2b1f2e555ebe" exitCode=0
Jul 28 17:57:29.640777 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.640717    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerDied","Data":"3082937b4bec5d3d876824a15e30b540e8bb2b9e6014bb59ba8a2b1f2e555ebe"}
Jul 28 17:57:29.642296 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.642269    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ltcl9" event={"ID":"c27ba9c5-86c1-4f79-a3d7-425d5c2d750f","Type":"ContainerStarted","Data":"3690e4f068ae1a0ee5c51e6f395779fdf16708d72e3a6bad40687bdc76f781d1"}
Jul 28 17:57:29.644089 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.644070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd" event={"ID":"3280d6cf-afd3-4d04-b0e7-72db473e4867","Type":"ContainerStarted","Data":"d05df71ad76c4829e53ed666179f902242be81ffcb7fb33b1ce20e66e3bf3710"}
Jul 28 17:57:29.645658 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.645567    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvwm8" event={"ID":"e1a7dab8-512b-4b50-bad4-6978e3e99045","Type":"ContainerStarted","Data":"c47d7652eb450487e296231abdb2c8e80251d9c7fb66843a56a2a863e9244704"}
Jul 28 17:57:29.647011 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.646967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqss8" event={"ID":"599dce4a-55e5-489e-bd51-5c4251e9c3ed","Type":"ContainerStarted","Data":"d25a325b2917a976be6723cf9968cb6238151f77fe7b03f80315455cb453e840"}
Jul 28 17:57:29.648448 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.648349    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8whqv" event={"ID":"d52be987-69c6-4133-a75b-6f5453ca3552","Type":"ContainerStarted","Data":"0ad07815550691ad4104d81e4a1045df84ac22b3b7eb6137c3362dafaa9cdc8e"}
Jul 28 17:57:29.651056 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.651035    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"2f209883313a2fa78c1efac3fd2b747eedd1c57f23542bc75d917e56dda3a53a"}
Jul 28 17:57:29.651155 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.651089    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"baa50cf5821e850a1bd4f778d28e840dfff0725caf7b44f93be531ef5cedb6df"}
Jul 28 17:57:29.651155 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.651118    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"03ec2e6512a3acbe28fd2b2b16b42c20a5110587878e792e7a44cfd5d89b7def"}
Jul 28 17:57:29.651155 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.651150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"7c0908dd13e6fb3b6abf9e1014526e6eede01c3886270b1f4d02aa0484b24229"}
Jul 28 17:57:29.660337 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.660301    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lsjq5" podStartSLOduration=3.037750069 podStartE2EDuration="22.660287026s" totalImagesPullingTime="19.622536957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:29.659817683 +0000 UTC m=+23.805370658" watchObservedRunningTime="2026-07-28 17:57:29.660287026 +0000 UTC m=+23.805840000"
Jul 28 17:57:29.660435 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.660418    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" podStartSLOduration=21.660414098 podStartE2EDuration="21.660414098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:11.619569852 +0000 UTC m=+5.765122830" watchObservedRunningTime="2026-07-28 17:57:29.660414098 +0000 UTC m=+23.805967072"
Jul 28 17:57:29.674030 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.673989    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ltcl9" podStartSLOduration=3.045962254 podStartE2EDuration="22.673975391s" totalImagesPullingTime="19.628013137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:29.67363118 +0000 UTC m=+23.819184154" watchObservedRunningTime="2026-07-28 17:57:29.673975391 +0000 UTC m=+23.819528356"
Jul 28 17:57:29.687370 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.687331    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zqss8" podStartSLOduration=3.055435822 podStartE2EDuration="22.687315377s" totalImagesPullingTime="19.631879555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:29.687242153 +0000 UTC m=+23.832795127" watchObservedRunningTime="2026-07-28 17:57:29.687315377 +0000 UTC m=+23.832868351"
Jul 28 17:57:29.705131 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.705088    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8whqv" podStartSLOduration=2.994955688 podStartE2EDuration="22.705071247s" totalImagesPullingTime="19.710115559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:29.704873959 +0000 UTC m=+23.850426929" watchObservedRunningTime="2026-07-28 17:57:29.705071247 +0000 UTC m=+23.850624218"
Jul 28 17:57:29.722578 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.722514    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nvwm8" podStartSLOduration=3.095736559 podStartE2EDuration="22.722496075s" totalImagesPullingTime="19.626759516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:29.722003188 +0000 UTC m=+23.867556161" watchObservedRunningTime="2026-07-28 17:57:29.722496075 +0000 UTC m=+23.868049049"
Jul 28 17:57:29.980631 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.980533    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:29.981182 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:29.981162    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:30.395057 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.395024    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:30.395220 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:30.395156    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:30.395453 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.395425    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:30.395576 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:30.395554    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:30.654913 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.654822    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rbrw2" event={"ID":"696d718f-48c0-48c8-aca4-a83421d1f85e","Type":"ContainerStarted","Data":"6887ec7d61ecd34b9f290093c7cb1b7a2067f05c061eadd869d007d2ca4c1861"}
Jul 28 17:57:30.659295 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.659252    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"0c832429692cf6286025d2dbd7922319860215c904b59a05ad99ab31a9693c54"}
Jul 28 17:57:30.659465 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.659336    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"f0c9d9e162f549250350d20ea92d32efb5e351238a654167102d1fc77d4989a4"}
Jul 28 17:57:30.659745 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.659679    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:30.660183 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.660163    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ltcl9"
Jul 28 17:57:30.671635 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.670239    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rbrw2" podStartSLOduration=4.042277251 podStartE2EDuration="23.670216657s" totalImagesPullingTime="19.627939406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:30.669903532 +0000 UTC m=+24.815456505" watchObservedRunningTime="2026-07-28 17:57:30.670216657 +0000 UTC m=+24.815769646"
Jul 28 17:57:30.897268 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:30.897236    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 17:57:31.331488 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:31.331345    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:57:30.897257333Z","UUID":"4babd676-3435-4fc9-b5b3-8d97e93d00cc","Handler":null,"Name":"","Endpoint":""}
Jul 28 17:57:31.333157 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:31.333130    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 17:57:31.333283 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:31.333170    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 17:57:31.394138 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:31.394099    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:31.394293 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:31.394220    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:31.664786 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:31.664715    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd" event={"ID":"3280d6cf-afd3-4d04-b0e7-72db473e4867","Type":"ContainerStarted","Data":"c4d808a36bb410fd3384915313b9f140be33316e8d6a16f10264e8cb38c3a433"}
Jul 28 17:57:32.394966 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:32.394886    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:32.395128 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:32.395029    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:32.395128 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:32.395083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:32.395242 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:32.395177    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:32.670228 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:32.670148    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"6bfbb3cdcdc50dbe31fc50fc2ff0d1767b8cfea50871ab02de62b73ae0c80e6f"}
Jul 28 17:57:33.394478 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:33.394445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:33.394663 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:33.394573    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:34.395118 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.394878    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:34.395849 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.394934    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:34.395849 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:34.395174    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:34.395849 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:34.395234    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:34.676026 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.675942    2575 generic.go:356] "Generic (PLEG): container finished" podID="bf5686ec-91c6-42a7-8436-1df336b0221c" containerID="39fb93072a17a1d60e3ce96747f58e67f88fbf8eee3959623971665d564213f3" exitCode=0
Jul 28 17:57:34.676026 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.675985    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerDied","Data":"39fb93072a17a1d60e3ce96747f58e67f88fbf8eee3959623971665d564213f3"}
Jul 28 17:57:34.678204 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.678176    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd" event={"ID":"3280d6cf-afd3-4d04-b0e7-72db473e4867","Type":"ContainerStarted","Data":"6ee75b948874036a4762e591c354e2d5cc3ad82f2e2f8f22c5a04447dcfedb82"}
Jul 28 17:57:34.681749 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.681726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" event={"ID":"84007fd6-0b1b-45cb-b78c-01cff9dca00d","Type":"ContainerStarted","Data":"2f07ed5165ef6c22998897fb7f5a1a90caf8bafa9917855c832e9d0485e82b66"}
Jul 28 17:57:34.682072 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.682045    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:34.698110 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.698086    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:34.722270 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.722214    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn" podStartSLOduration=8.001990124 podStartE2EDuration="27.722196218s" totalImagesPullingTime="19.720206094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:34.721055631 +0000 UTC m=+28.866608605" watchObservedRunningTime="2026-07-28 17:57:34.722196218 +0000 UTC m=+28.867749193"
Jul 28 17:57:34.738331 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:34.738276    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qc2rd" podStartSLOduration=3.775965652 podStartE2EDuration="27.738257666s" totalImagesPullingTime="23.962292014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:34.737844803 +0000 UTC m=+28.883397788" watchObservedRunningTime="2026-07-28 17:57:34.738257666 +0000 UTC m=+28.883810639"
Jul 28 17:57:35.394504 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:35.394466    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:35.394794 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:35.394609    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:35.684706 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:35.684610    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:35.684706 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:35.684683    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:35.700768 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:35.700742    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:57:36.199615 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.199166    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vdwk2"]
Jul 28 17:57:36.199615 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.199594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:36.199834 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:36.199704    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:36.200014 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.199989    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q8q24"]
Jul 28 17:57:36.200158 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.200094    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:36.200234 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:36.200180    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:36.200587 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.200564    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jck5m"]
Jul 28 17:57:36.200690 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.200674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:36.200787 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:36.200766    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:36.688341 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.688253    2575 generic.go:356] "Generic (PLEG): container finished" podID="bf5686ec-91c6-42a7-8436-1df336b0221c" containerID="2448680fab100e9efa02b360af6cb778de49b614bc2dc526aae6a19d9839044e" exitCode=0
Jul 28 17:57:36.688812 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:36.688337    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerDied","Data":"2448680fab100e9efa02b360af6cb778de49b614bc2dc526aae6a19d9839044e"}
Jul 28 17:57:38.394442 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:38.394401    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:38.394947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:38.394410    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:38.394947 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:38.394511    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:38.394947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:38.394523    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:38.394947 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:38.394612    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:38.394947 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:38.394681    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:38.695802 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:38.695720    2575 generic.go:356] "Generic (PLEG): container finished" podID="bf5686ec-91c6-42a7-8436-1df336b0221c" containerID="a3dd8e5be1d8dce44699217999d239bda38c6762e34dccbdefca531483657d3d" exitCode=0
Jul 28 17:57:38.695802 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:38.695770    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerDied","Data":"a3dd8e5be1d8dce44699217999d239bda38c6762e34dccbdefca531483657d3d"}
Jul 28 17:57:40.007491 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:40.007290    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:40.007491 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.007479    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:40.008000 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.007566    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs podName:5f6924da-d133-4898-8aa4-0a1aa892ff2e nodeName:}" failed. No retries permitted until 2026-07-28 17:58:12.007545428 +0000 UTC m=+66.153098393 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs") pod "network-metrics-daemon-jck5m" (UID: "5f6924da-d133-4898-8aa4-0a1aa892ff2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:40.108229 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:40.108184    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:40.108229 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:40.108237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:40.108459 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.108356    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:40.108459 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.108424    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret podName:55e27a0d-949a-44da-9368-a62947420498 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:12.108409087 +0000 UTC m=+66.253962040 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret") pod "global-pull-secret-syncer-vdwk2" (UID: "55e27a0d-949a-44da-9368-a62947420498") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:40.108459 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.108364    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:40.108641 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.108477    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:40.108641 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.108490    2575 projected.go:196] Error preparing data for projected volume kube-api-access-gbwr8 for pod openshift-network-diagnostics/network-check-target-q8q24: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:40.108641 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.108532    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8 podName:724f5f04-2dc2-4aca-8088-812f03ebf9f8 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:12.108521461 +0000 UTC m=+66.254074414 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gbwr8" (UniqueName: "kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8") pod "network-check-target-q8q24" (UID: "724f5f04-2dc2-4aca-8088-812f03ebf9f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:40.394845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:40.394765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:40.394845 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:40.394797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:40.395095 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:40.394766    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:40.395095 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.394923    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jck5m" podUID="5f6924da-d133-4898-8aa4-0a1aa892ff2e"
Jul 28 17:57:40.395095 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.395017    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdwk2" podUID="55e27a0d-949a-44da-9368-a62947420498"
Jul 28 17:57:40.395095 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:40.395083    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q8q24" podUID="724f5f04-2dc2-4aca-8088-812f03ebf9f8"
Jul 28 17:57:42.171178 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.171146    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeReady"
Jul 28 17:57:42.171803 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.171306    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 17:57:42.228530 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.228496    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jd452"]
Jul 28 17:57:42.231832 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.231804    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tsm5b"]
Jul 28 17:57:42.232025 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.231950    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.234180 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.234150    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.234720 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.234697    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 17:57:42.234720 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.234711    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 17:57:42.234947 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.234806    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5mqwj\""
Jul 28 17:57:42.236716 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.236695    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 17:57:42.236833 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.236758    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 17:57:42.236833 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.236784    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vjgn2\""
Jul 28 17:57:42.237022 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.237006    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 17:57:42.245439 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.245403    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jd452"]
Jul 28 17:57:42.260838 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.260807    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tsm5b"]
Jul 28 17:57:42.324242 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.324205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtrql\" (UniqueName: \"kubernetes.io/projected/976d2309-867a-4a03-a1c1-fc52341a4c05-kube-api-access-xtrql\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.324242 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.324256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6lfj\" (UniqueName: \"kubernetes.io/projected/4d687f14-4aab-4083-b44d-1ed34c9a051c-kube-api-access-d6lfj\") pod \"ingress-canary-tsm5b\" (UID: \"4d687f14-4aab-4083-b44d-1ed34c9a051c\") " pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.324488 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.324281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.324488 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.324347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/976d2309-867a-4a03-a1c1-fc52341a4c05-tmp-dir\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.324488 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.324419    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/976d2309-867a-4a03-a1c1-fc52341a4c05-config-volume\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.324488 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.324474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d687f14-4aab-4083-b44d-1ed34c9a051c-cert\") pod \"ingress-canary-tsm5b\" (UID: \"4d687f14-4aab-4083-b44d-1ed34c9a051c\") " pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.394715 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.394681    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:57:42.394715 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.394713    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:57:42.394976 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.394681    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:57:42.397414 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.397390    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:57:42.397585 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.397444    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:57:42.397662 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.397610    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pbl4v\""
Jul 28 17:57:42.397662 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.397617    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:57:42.397662 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.397638    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:57:42.397832 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.397752    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgwbs\""
Jul 28 17:57:42.425274 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.425198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6lfj\" (UniqueName: \"kubernetes.io/projected/4d687f14-4aab-4083-b44d-1ed34c9a051c-kube-api-access-d6lfj\") pod \"ingress-canary-tsm5b\" (UID: \"4d687f14-4aab-4083-b44d-1ed34c9a051c\") " pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.425274 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.425231    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.425274 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.425249    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/976d2309-867a-4a03-a1c1-fc52341a4c05-tmp-dir\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.425541 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.425326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/976d2309-867a-4a03-a1c1-fc52341a4c05-config-volume\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.425541 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.425393    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d687f14-4aab-4083-b44d-1ed34c9a051c-cert\") pod \"ingress-canary-tsm5b\" (UID: \"4d687f14-4aab-4083-b44d-1ed34c9a051c\") " pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.425541 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.425447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtrql\" (UniqueName: \"kubernetes.io/projected/976d2309-867a-4a03-a1c1-fc52341a4c05-kube-api-access-xtrql\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.425697 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.425590    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/976d2309-867a-4a03-a1c1-fc52341a4c05-tmp-dir\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.425697 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:42.425644    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:42.425796 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:42.425708    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls podName:976d2309-867a-4a03-a1c1-fc52341a4c05 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:42.925688201 +0000 UTC m=+37.071241163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls") pod "dns-default-jd452" (UID: "976d2309-867a-4a03-a1c1-fc52341a4c05") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:42.426087 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.426067    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/976d2309-867a-4a03-a1c1-fc52341a4c05-config-volume\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.430117 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.430096    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d687f14-4aab-4083-b44d-1ed34c9a051c-cert\") pod \"ingress-canary-tsm5b\" (UID: \"4d687f14-4aab-4083-b44d-1ed34c9a051c\") " pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.434930 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.434907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtrql\" (UniqueName: \"kubernetes.io/projected/976d2309-867a-4a03-a1c1-fc52341a4c05-kube-api-access-xtrql\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.435134 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.435105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6lfj\" (UniqueName: \"kubernetes.io/projected/4d687f14-4aab-4083-b44d-1ed34c9a051c-kube-api-access-d6lfj\") pod \"ingress-canary-tsm5b\" (UID: \"4d687f14-4aab-4083-b44d-1ed34c9a051c\") " pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.551843 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.551793    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tsm5b"
Jul 28 17:57:42.695775 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.695452    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tsm5b"]
Jul 28 17:57:42.699115 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:42.699078    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d687f14_4aab_4083_b44d_1ed34c9a051c.slice/crio-2d160b9194c58099e8bb0c3bdfad395a23724435bc55a110ea4621a3237e104a WatchSource:0}: Error finding container 2d160b9194c58099e8bb0c3bdfad395a23724435bc55a110ea4621a3237e104a: Status 404 returned error can't find the container with id 2d160b9194c58099e8bb0c3bdfad395a23724435bc55a110ea4621a3237e104a
Jul 28 17:57:42.706756 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.706724    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tsm5b" event={"ID":"4d687f14-4aab-4083-b44d-1ed34c9a051c","Type":"ContainerStarted","Data":"2d160b9194c58099e8bb0c3bdfad395a23724435bc55a110ea4621a3237e104a"}
Jul 28 17:57:42.709581 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.709554    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerStarted","Data":"ff18a3d7d9aaa0f034bb8e756fff3b7af1c530b9a153d66aabd9b6a9dba7a157"}
Jul 28 17:57:42.930130 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:42.930087    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:42.930302 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:42.930206    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:42.930302 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:42.930261    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls podName:976d2309-867a-4a03-a1c1-fc52341a4c05 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:43.930248351 +0000 UTC m=+38.075801309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls") pod "dns-default-jd452" (UID: "976d2309-867a-4a03-a1c1-fc52341a4c05") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:43.718472 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:43.718428    2575 generic.go:356] "Generic (PLEG): container finished" podID="bf5686ec-91c6-42a7-8436-1df336b0221c" containerID="ff18a3d7d9aaa0f034bb8e756fff3b7af1c530b9a153d66aabd9b6a9dba7a157" exitCode=0
Jul 28 17:57:43.719324 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:43.718501    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerDied","Data":"ff18a3d7d9aaa0f034bb8e756fff3b7af1c530b9a153d66aabd9b6a9dba7a157"}
Jul 28 17:57:43.939396 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:43.939354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:43.939590 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:43.939520    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:43.939656 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:43.939613    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls podName:976d2309-867a-4a03-a1c1-fc52341a4c05 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:45.939593476 +0000 UTC m=+40.085146443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls") pod "dns-default-jd452" (UID: "976d2309-867a-4a03-a1c1-fc52341a4c05") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:44.724445 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:44.724407    2575 generic.go:356] "Generic (PLEG): container finished" podID="bf5686ec-91c6-42a7-8436-1df336b0221c" containerID="1a898ccf29008c400bd3d67bc8bf1c92a85b9a0daff924e2d82aeb1b6ce7b8d1" exitCode=0
Jul 28 17:57:44.724904 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:44.724457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerDied","Data":"1a898ccf29008c400bd3d67bc8bf1c92a85b9a0daff924e2d82aeb1b6ce7b8d1"}
Jul 28 17:57:45.728152 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:45.728115    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tsm5b" event={"ID":"4d687f14-4aab-4083-b44d-1ed34c9a051c","Type":"ContainerStarted","Data":"161816e972ef4edb7038ad8cb679a4ba286675d24e0bc73c407854adbd400b0e"}
Jul 28 17:57:45.731768 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:45.731526    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" event={"ID":"bf5686ec-91c6-42a7-8436-1df336b0221c","Type":"ContainerStarted","Data":"b9da1f4c5262c9eed403376c7fb2ef44bc9d64fca8387427a441d263d1fee2e3"}
Jul 28 17:57:45.743529 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:45.743470    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tsm5b" podStartSLOduration=1.380901777 podStartE2EDuration="3.743450525s" totalImagesPullingTime="2.362548748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:45.743237622 +0000 UTC m=+39.888790598" watchObservedRunningTime="2026-07-28 17:57:45.743450525 +0000 UTC m=+39.889003501"
Jul 28 17:57:45.766193 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:45.766144    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xs4kr" podStartSLOduration=8.90050613 podStartE2EDuration="38.766127963s" totalImagesPullingTime="29.865621833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:45.765482212 +0000 UTC m=+39.911035187" watchObservedRunningTime="2026-07-28 17:57:45.766127963 +0000 UTC m=+39.911680937"
Jul 28 17:57:45.960337 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:45.960295    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:45.960513 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:45.960439    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:45.960513 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:45.960499    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls podName:976d2309-867a-4a03-a1c1-fc52341a4c05 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:49.960483188 +0000 UTC m=+44.106036161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls") pod "dns-default-jd452" (UID: "976d2309-867a-4a03-a1c1-fc52341a4c05") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:48.575749 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.575711    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-75c4fdbf55-r9lmg"]
Jul 28 17:57:48.606765 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.606722    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75c4fdbf55-r9lmg"]
Jul 28 17:57:48.606957 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.606845    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.609952 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.609921    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 17:57:48.609952 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.609946    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 17:57:48.610164 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.609959    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2vjht\""
Jul 28 17:57:48.610407 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.610385    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 17:57:48.610536 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.610423    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 17:57:48.610536 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.610483    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 17:57:48.610936 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.610919    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 17:57:48.675761 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.675726    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fbhfk"]
Jul 28 17:57:48.697534 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.697504    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"]
Jul 28 17:57:48.697698 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.697628    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.700025 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.699981    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:48.700025 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.700023    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 17:57:48.700294 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.700025    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 17:57:48.700294 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.700230    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:48.700713 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.700677    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hrh8q\""
Jul 28 17:57:48.707605 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.707583    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 17:57:48.713522 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.713491    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"]
Jul 28 17:57:48.713656 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.713602    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:48.715889 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.715844    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 17:57:48.716012 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.715867    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hrl8c\""
Jul 28 17:57:48.716012 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.715905    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:48.716012 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.715960    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:48.735551 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.735523    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"]
Jul 28 17:57:48.735551 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.735558    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"]
Jul 28 17:57:48.735756 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.735573    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fbhfk"]
Jul 28 17:57:48.735756 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.735656    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.738545 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.738519    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 17:57:48.738545 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.738519    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 17:57:48.738744 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.738566    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:48.738744 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.738631    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:48.738744 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.738662    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zqb2b\""
Jul 28 17:57:48.763328 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.763294    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"]
Jul 28 17:57:48.779605 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.779577    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"]
Jul 28 17:57:48.779750 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.779676    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:48.782060 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb6fq\" (UniqueName: \"kubernetes.io/projected/c1e7e625-fd4c-4c3a-8c40-798c157f5173-kube-api-access-bb6fq\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.782181 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782076    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-config\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.782181 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782110    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-trusted-ca\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.782181 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782125    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 17:57:48.782181 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782171    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.782379 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782193    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-default-certificate\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.782379 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782125    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9g2gw\""
Jul 28 17:57:48.782379 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782115    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 17:57:48.782379 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782218    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gts74\" (UniqueName: \"kubernetes.io/projected/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-kube-api-access-gts74\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.782379 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782261    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-stats-auth\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.782379 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.782379 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.782364    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-serving-cert\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.883287 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.883287 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-default-certificate\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.883287 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883260    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gts74\" (UniqueName: \"kubernetes.io/projected/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-kube-api-access-gts74\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.883287 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-stats-auth\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883318    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:48.883375    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:49.383355525 +0000 UTC m=+43.528908477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : configmap references non-existent config key: service-ca.crt
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883446    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h74l6\" (UniqueName: \"kubernetes.io/projected/84d25708-bdbe-40a4-ba8b-ee1d860e260d-kube-api-access-h74l6\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrdst\" (UniqueName: \"kubernetes.io/projected/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-kube-api-access-qrdst\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.883560 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883499    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.884127 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883581    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.884127 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883614    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-serving-cert\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.884127 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:48.883667    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:57:48.884127 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb6fq\" (UniqueName: \"kubernetes.io/projected/c1e7e625-fd4c-4c3a-8c40-798c157f5173-kube-api-access-bb6fq\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.884127 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883709    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-config\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.884127 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:48.883737    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:49.38371745 +0000 UTC m=+43.529270414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : secret "router-metrics-certs-default" not found
Jul 28 17:57:48.884127 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.883776    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-trusted-ca\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.884461 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.884367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-config\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.884761 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.884738    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-trusted-ca\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.887168 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.887142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-serving-cert\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.887317 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.887300    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-stats-auth\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.887422 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.887402    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-default-certificate\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.892490 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.892461    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gts74\" (UniqueName: \"kubernetes.io/projected/4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef-kube-api-access-gts74\") pod \"console-operator-584dcb6b54-fbhfk\" (UID: \"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef\") " pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:48.892747 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.892723    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb6fq\" (UniqueName: \"kubernetes.io/projected/c1e7e625-fd4c-4c3a-8c40-798c157f5173-kube-api-access-bb6fq\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:48.984591 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.984555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:48.984591 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.984592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:48.984591 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.984610    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.984847 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.984629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:48.984847 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.984652    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h74l6\" (UniqueName: \"kubernetes.io/projected/84d25708-bdbe-40a4-ba8b-ee1d860e260d-kube-api-access-h74l6\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:48.984847 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:48.984713    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 17:57:48.984847 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:48.984730    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:57:48.984847 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:48.984786    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert podName:72a6a8a7-dc6e-4b60-b06f-6f6c3497307d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:49.484765341 +0000 UTC m=+43.630318317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pldz2" (UID: "72a6a8a7-dc6e-4b60-b06f-6f6c3497307d") : secret "networking-console-plugin-cert" not found
Jul 28 17:57:48.984847 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:48.984813    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls podName:84d25708-bdbe-40a4-ba8b-ee1d860e260d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:49.484803323 +0000 UTC m=+43.630356281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls") pod "cluster-samples-operator-6748489656-wns8z" (UID: "84d25708-bdbe-40a4-ba8b-ee1d860e260d") : secret "samples-operator-tls" not found
Jul 28 17:57:48.985139 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.984879    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrdst\" (UniqueName: \"kubernetes.io/projected/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-kube-api-access-qrdst\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.985139 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.984914    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.985395 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.985377    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:48.985395 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.985388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.986837 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.986820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.993581 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.993553    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrdst\" (UniqueName: \"kubernetes.io/projected/f0fd56f1-1ed2-4ffc-b730-b575cfe7380e-kube-api-access-qrdst\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc\" (UID: \"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:48.994033 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:48.994014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h74l6\" (UniqueName: \"kubernetes.io/projected/84d25708-bdbe-40a4-ba8b-ee1d860e260d-kube-api-access-h74l6\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:49.009248 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.009209    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:49.044298 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.044272    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"
Jul 28 17:57:49.175266 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.175233    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fbhfk"]
Jul 28 17:57:49.178781 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:49.178752    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7dc8b4_ebdd_47ae_9ef9_77a89f7ba0ef.slice/crio-df84f2398933435d23a1169b3f254d54d89674f6b95b7c80c29e7fe73c597d17 WatchSource:0}: Error finding container df84f2398933435d23a1169b3f254d54d89674f6b95b7c80c29e7fe73c597d17: Status 404 returned error can't find the container with id df84f2398933435d23a1169b3f254d54d89674f6b95b7c80c29e7fe73c597d17
Jul 28 17:57:49.197543 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.197517    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc"]
Jul 28 17:57:49.201043 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:57:49.201013    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fd56f1_1ed2_4ffc_b730_b575cfe7380e.slice/crio-b5aff57e3cb91c2ff7b5bfa9acb250ad91d52ca9eb7c3d480306712a3d3fb437 WatchSource:0}: Error finding container b5aff57e3cb91c2ff7b5bfa9acb250ad91d52ca9eb7c3d480306712a3d3fb437: Status 404 returned error can't find the container with id b5aff57e3cb91c2ff7b5bfa9acb250ad91d52ca9eb7c3d480306712a3d3fb437
Jul 28 17:57:49.387533 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.387503    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:49.387700 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.387586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:49.387700 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.387681    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:50.387663254 +0000 UTC m=+44.533216211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : configmap references non-existent config key: service-ca.crt
Jul 28 17:57:49.387780 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.387702    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:57:49.387780 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.387745    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:50.387730593 +0000 UTC m=+44.533283551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : secret "router-metrics-certs-default" not found
Jul 28 17:57:49.489030 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.488932    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:49.489212 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.489069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:49.489212 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.489095    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:57:49.489212 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.489166    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 17:57:49.489212 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.489179    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls podName:84d25708-bdbe-40a4-ba8b-ee1d860e260d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:50.489156517 +0000 UTC m=+44.634709484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls") pod "cluster-samples-operator-6748489656-wns8z" (UID: "84d25708-bdbe-40a4-ba8b-ee1d860e260d") : secret "samples-operator-tls" not found
Jul 28 17:57:49.489212 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.489204    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert podName:72a6a8a7-dc6e-4b60-b06f-6f6c3497307d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:50.489194142 +0000 UTC m=+44.634747098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pldz2" (UID: "72a6a8a7-dc6e-4b60-b06f-6f6c3497307d") : secret "networking-console-plugin-cert" not found
Jul 28 17:57:49.744268 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.744237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc" event={"ID":"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e","Type":"ContainerStarted","Data":"b5aff57e3cb91c2ff7b5bfa9acb250ad91d52ca9eb7c3d480306712a3d3fb437"}
Jul 28 17:57:49.745337 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.745313    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" event={"ID":"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef","Type":"ContainerStarted","Data":"df84f2398933435d23a1169b3f254d54d89674f6b95b7c80c29e7fe73c597d17"}
Jul 28 17:57:49.992957 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:49.992918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:49.993156 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.993114    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:49.993243 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:49.993201    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls podName:976d2309-867a-4a03-a1c1-fc52341a4c05 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:57.993177911 +0000 UTC m=+52.138730881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls") pod "dns-default-jd452" (UID: "976d2309-867a-4a03-a1c1-fc52341a4c05") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:50.396341 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:50.396300    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:50.396525 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:50.396415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:50.396525 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:50.396480    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:57:50.396639 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:50.396556    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:52.396536583 +0000 UTC m=+46.542089543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : secret "router-metrics-certs-default" not found
Jul 28 17:57:50.396639 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:50.396574    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:52.396568067 +0000 UTC m=+46.542121022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : configmap references non-existent config key: service-ca.crt
Jul 28 17:57:50.497618 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:50.497577    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:50.497618 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:50.497625    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:50.497846 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:50.497731    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 17:57:50.497846 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:50.497775    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:57:50.497846 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:50.497790    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert podName:72a6a8a7-dc6e-4b60-b06f-6f6c3497307d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:52.497776809 +0000 UTC m=+46.643329765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pldz2" (UID: "72a6a8a7-dc6e-4b60-b06f-6f6c3497307d") : secret "networking-console-plugin-cert" not found
Jul 28 17:57:50.497846 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:50.497831    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls podName:84d25708-bdbe-40a4-ba8b-ee1d860e260d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:52.497816211 +0000 UTC m=+46.643369163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls") pod "cluster-samples-operator-6748489656-wns8z" (UID: "84d25708-bdbe-40a4-ba8b-ee1d860e260d") : secret "samples-operator-tls" not found
Jul 28 17:57:52.414627 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.414551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:52.415209 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.414695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:52.415209 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:52.414835    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.414815601 +0000 UTC m=+50.560368573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : configmap references non-existent config key: service-ca.crt
Jul 28 17:57:52.415209 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:52.414938    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:57:52.415209 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:52.414975    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.414963261 +0000 UTC m=+50.560516216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : secret "router-metrics-certs-default" not found
Jul 28 17:57:52.516031 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.515945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:52.516031 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.515990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:52.516223 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:52.516106    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:57:52.516223 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:52.516113    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 17:57:52.516223 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:52.516161    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls podName:84d25708-bdbe-40a4-ba8b-ee1d860e260d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.516146629 +0000 UTC m=+50.661699586 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls") pod "cluster-samples-operator-6748489656-wns8z" (UID: "84d25708-bdbe-40a4-ba8b-ee1d860e260d") : secret "samples-operator-tls" not found
Jul 28 17:57:52.516330 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:52.516228    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert podName:72a6a8a7-dc6e-4b60-b06f-6f6c3497307d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.516221082 +0000 UTC m=+50.661774034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pldz2" (UID: "72a6a8a7-dc6e-4b60-b06f-6f6c3497307d") : secret "networking-console-plugin-cert" not found
Jul 28 17:57:52.755388 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.755347    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc" event={"ID":"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e","Type":"ContainerStarted","Data":"6fea10078bb0a06cee3977ca7f386fcd337ac634c7919a895af3b47bb8baa40e"}
Jul 28 17:57:52.756995 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.756971    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/0.log"
Jul 28 17:57:52.757131 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.757016    2575 generic.go:356] "Generic (PLEG): container finished" podID="4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef" containerID="8dfaa8d78f6b5b477bc415338a1d5f9654d07a36abdc0897049ca7fd699ecc56" exitCode=255
Jul 28 17:57:52.757131 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.757047    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" event={"ID":"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef","Type":"ContainerDied","Data":"8dfaa8d78f6b5b477bc415338a1d5f9654d07a36abdc0897049ca7fd699ecc56"}
Jul 28 17:57:52.757313 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.757296    2575 scope.go:117] "RemoveContainer" containerID="8dfaa8d78f6b5b477bc415338a1d5f9654d07a36abdc0897049ca7fd699ecc56"
Jul 28 17:57:52.772424 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:52.772367    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc" podStartSLOduration=1.697906044 podStartE2EDuration="4.772345992s" totalImagesPullingTime="3.074439948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:52.771489216 +0000 UTC m=+46.917042190" watchObservedRunningTime="2026-07-28 17:57:52.772345992 +0000 UTC m=+46.917898959"
Jul 28 17:57:53.761837 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:53.761809    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 17:57:53.762269 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:53.762250    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/0.log"
Jul 28 17:57:53.762327 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:53.762295    2575 generic.go:356] "Generic (PLEG): container finished" podID="4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef" containerID="59b8f119c3f40409095a8fac9ff2898075dff6ff1a97ec61b1f3226a1944da7a" exitCode=255
Jul 28 17:57:53.762420 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:53.762397    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" event={"ID":"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef","Type":"ContainerDied","Data":"59b8f119c3f40409095a8fac9ff2898075dff6ff1a97ec61b1f3226a1944da7a"}
Jul 28 17:57:53.775556 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:53.762456    2575 scope.go:117] "RemoveContainer" containerID="8dfaa8d78f6b5b477bc415338a1d5f9654d07a36abdc0897049ca7fd699ecc56"
Jul 28 17:57:53.775665 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:53.775646    2575 scope.go:117] "RemoveContainer" containerID="59b8f119c3f40409095a8fac9ff2898075dff6ff1a97ec61b1f3226a1944da7a"
Jul 28 17:57:53.775934 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:53.775897    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fbhfk_openshift-console-operator(4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" podUID="4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef"
Jul 28 17:57:54.766396 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:54.766369    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 17:57:54.766796 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:54.766709    2575 scope.go:117] "RemoveContainer" containerID="59b8f119c3f40409095a8fac9ff2898075dff6ff1a97ec61b1f3226a1944da7a"
Jul 28 17:57:54.766902 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:54.766879    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fbhfk_openshift-console-operator(4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" podUID="4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef"
Jul 28 17:57:55.183578 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.183483    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-79fdg"]
Jul 28 17:57:55.218895 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.218837    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-79fdg"]
Jul 28 17:57:55.219045 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.218968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.221730 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.221699    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 17:57:55.221730 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.221700    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 17:57:55.222102 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.222087    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 17:57:55.222170 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.222123    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 17:57:55.222825 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.222798    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7kkv\""
Jul 28 17:57:55.340438 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.340391    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4bb9ff-f8d8-445e-b551-eb10ad499789-data-volume\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.340438 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.340440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62g4k\" (UniqueName: \"kubernetes.io/projected/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-api-access-62g4k\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.340619 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.340560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.340619 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.340609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4f4bb9ff-f8d8-445e-b551-eb10ad499789-crio-socket\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.340704 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.340657    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.441187 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.441097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.441187 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.441156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4f4bb9ff-f8d8-445e-b551-eb10ad499789-crio-socket\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.441363 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.441262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.441363 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.441270    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:55.441363 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.441297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4bb9ff-f8d8-445e-b551-eb10ad499789-data-volume\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.441363 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.441354    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:55.941334194 +0000 UTC m=+50.086887149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:55.441520 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.441369    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.441520 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.441432    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:55.941415768 +0000 UTC m=+50.086968735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.441520 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.441460    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62g4k\" (UniqueName: \"kubernetes.io/projected/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-api-access-62g4k\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.441520 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.441486    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4f4bb9ff-f8d8-445e-b551-eb10ad499789-crio-socket\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.441642 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.441583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4bb9ff-f8d8-445e-b551-eb10ad499789-data-volume\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.453397 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.453367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62g4k\" (UniqueName: \"kubernetes.io/projected/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-api-access-62g4k\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.946562 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.946518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.946985 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.946644    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.946985 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:55.946661    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:55.946985 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.946712    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.946692682 +0000 UTC m=+51.092245658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.946985 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.946763    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:55.946985 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:55.946810    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.946798759 +0000 UTC m=+51.092351730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:56.451138 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:56.451095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:56.451337 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:56.451164    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:57:56.451337 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.451288    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:04.45126924 +0000 UTC m=+58.596822193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : configmap references non-existent config key: service-ca.crt
Jul 28 17:57:56.451337 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.451319    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:57:56.451492 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.451385    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:04.451371541 +0000 UTC m=+58.596924499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : secret "router-metrics-certs-default" not found
Jul 28 17:57:56.552204 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:56.552158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:57:56.552378 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:56.552262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:57:56.552378 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.552305    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 17:57:56.552378 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.552366    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls podName:84d25708-bdbe-40a4-ba8b-ee1d860e260d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:04.552351086 +0000 UTC m=+58.697904044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls") pod "cluster-samples-operator-6748489656-wns8z" (UID: "84d25708-bdbe-40a4-ba8b-ee1d860e260d") : secret "samples-operator-tls" not found
Jul 28 17:57:56.552497 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.552396    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 17:57:56.552497 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.552459    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert podName:72a6a8a7-dc6e-4b60-b06f-6f6c3497307d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:04.55244609 +0000 UTC m=+58.697999046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pldz2" (UID: "72a6a8a7-dc6e-4b60-b06f-6f6c3497307d") : secret "networking-console-plugin-cert" not found
Jul 28 17:57:56.955868 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:56.955821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:56.956237 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:56.955943    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:56.956237 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.955988    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:56.956237 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.956059    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:56.956237 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.956056    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:58.956041814 +0000 UTC m=+53.101594785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:56.956237 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:56.956129    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:58.956115125 +0000 UTC m=+53.101668078 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:58.065071 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:58.065033    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:57:58.065530 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:58.065193    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:58.065530 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:58.065257    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls podName:976d2309-867a-4a03-a1c1-fc52341a4c05 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:14.06524275 +0000 UTC m=+68.210795706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls") pod "dns-default-jd452" (UID: "976d2309-867a-4a03-a1c1-fc52341a4c05") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:58.247774 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:58.247742    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqss8_599dce4a-55e5-489e-bd51-5c4251e9c3ed/dns-node-resolver/0.log"
Jul 28 17:57:58.847814 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:58.847783    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvwm8_e1a7dab8-512b-4b50-bad4-6978e3e99045/node-ca/0.log"
Jul 28 17:57:58.971714 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:58.971674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:58.971930 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:58.971768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:57:58.971930 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:58.971836    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:58.971930 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:58.971902    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:58.971930 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:58.971917    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:02.971901294 +0000 UTC m=+57.117454250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:58.972100 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:58.971951    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:02.971938218 +0000 UTC m=+57.117491174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:59.009619 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:59.009580    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:59.009734 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:59.009638    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:57:59.010019 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:57:59.010006    2575 scope.go:117] "RemoveContainer" containerID="59b8f119c3f40409095a8fac9ff2898075dff6ff1a97ec61b1f3226a1944da7a"
Jul 28 17:57:59.010172 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:57:59.010150    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fbhfk_openshift-console-operator(4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" podUID="4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef"
Jul 28 17:58:00.050186 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:00.050156    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tsm5b_4d687f14-4aab-4083-b44d-1ed34c9a051c/serve-healthcheck-canary/0.log"
Jul 28 17:58:00.647069 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:00.647038    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc_f0fd56f1-1ed2-4ffc-b730-b575cfe7380e/kube-storage-version-migrator-operator/0.log"
Jul 28 17:58:03.006554 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:03.006509    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:58:03.007019 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:03.006645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:58:03.007019 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:03.006659    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:58:03.007019 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:03.006726    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:11.006709988 +0000 UTC m=+65.152262945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : configmap "kube-rbac-proxy" not found
Jul 28 17:58:03.007019 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:03.006788    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:58:03.007019 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:03.006841    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls podName:4f4bb9ff-f8d8-445e-b551-eb10ad499789 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:11.006827523 +0000 UTC m=+65.152380479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls") pod "insights-runtime-extractor-79fdg" (UID: "4f4bb9ff-f8d8-445e-b551-eb10ad499789") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:58:04.519351 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:04.519292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:04.519805 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:04.519436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:04.519805 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:04.519448    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 17:58:04.519805 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:04.519509    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:20.51949403 +0000 UTC m=+74.665046987 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : secret "router-metrics-certs-default" not found
Jul 28 17:58:04.519805 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:04.519579    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle podName:c1e7e625-fd4c-4c3a-8c40-798c157f5173 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:20.519565796 +0000 UTC m=+74.665118752 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle") pod "router-default-75c4fdbf55-r9lmg" (UID: "c1e7e625-fd4c-4c3a-8c40-798c157f5173") : configmap references non-existent config key: service-ca.crt
Jul 28 17:58:04.620782 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:04.620744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:58:04.621018 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:04.620816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:58:04.621018 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:04.620986    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 17:58:04.621137 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:04.621051    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert podName:72a6a8a7-dc6e-4b60-b06f-6f6c3497307d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:20.621032787 +0000 UTC m=+74.766585762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pldz2" (UID: "72a6a8a7-dc6e-4b60-b06f-6f6c3497307d") : secret "networking-console-plugin-cert" not found
Jul 28 17:58:04.623283 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:04.623258    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84d25708-bdbe-40a4-ba8b-ee1d860e260d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-wns8z\" (UID: \"84d25708-bdbe-40a4-ba8b-ee1d860e260d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:58:04.923035 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:04.922935    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"
Jul 28 17:58:05.055303 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:05.055265    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z"]
Jul 28 17:58:05.793018 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:05.792980    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z" event={"ID":"84d25708-bdbe-40a4-ba8b-ee1d860e260d","Type":"ContainerStarted","Data":"49aa0caad8dd03ee8463038523a758fdea6c5770e80b359bbe53d556393e7a97"}
Jul 28 17:58:07.711986 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:07.711956    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88wzn"
Jul 28 17:58:07.799397 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:07.799344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z" event={"ID":"84d25708-bdbe-40a4-ba8b-ee1d860e260d","Type":"ContainerStarted","Data":"c9e3c813b78d6df0a97b48a4f07a170c6ab23d40f67de35fbe5e52e989e93162"}
Jul 28 17:58:07.799574 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:07.799413    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z" event={"ID":"84d25708-bdbe-40a4-ba8b-ee1d860e260d","Type":"ContainerStarted","Data":"0a81391b33cffdf29466741c38dc2efda2e2e5cc3d2bc8defd5a485b263d3b4d"}
Jul 28 17:58:07.822506 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:07.822447    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-wns8z" podStartSLOduration=17.916357924 podStartE2EDuration="19.82242918s" totalImagesPullingTime="1.906071256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:07.821690094 +0000 UTC m=+61.967243069" watchObservedRunningTime="2026-07-28 17:58:07.82242918 +0000 UTC m=+61.967982195"
Jul 28 17:58:11.081507 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.081457    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:58:11.082011 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.081663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:58:11.082161 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.082144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4f4bb9ff-f8d8-445e-b551-eb10ad499789-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:58:11.083806 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.083787    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4f4bb9ff-f8d8-445e-b551-eb10ad499789-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-79fdg\" (UID: \"4f4bb9ff-f8d8-445e-b551-eb10ad499789\") " pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:58:11.130243 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.130211    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7kkv\""
Jul 28 17:58:11.138227 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.138196    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-79fdg"
Jul 28 17:58:11.271572 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.271542    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-79fdg"]
Jul 28 17:58:11.274675 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:11.274649    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4bb9ff_f8d8_445e_b551_eb10ad499789.slice/crio-4546a2f0351d9ca100b6598debd0063380e3274e1b2873f3ac7e7eee0a43d344 WatchSource:0}: Error finding container 4546a2f0351d9ca100b6598debd0063380e3274e1b2873f3ac7e7eee0a43d344: Status 404 returned error can't find the container with id 4546a2f0351d9ca100b6598debd0063380e3274e1b2873f3ac7e7eee0a43d344
Jul 28 17:58:11.394949 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.394844    2575 scope.go:117] "RemoveContainer" containerID="59b8f119c3f40409095a8fac9ff2898075dff6ff1a97ec61b1f3226a1944da7a"
Jul 28 17:58:11.814916 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.814658    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 17:58:11.814916 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.814765    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" event={"ID":"4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef","Type":"ContainerStarted","Data":"612e86006aecfb93869357477be20c7cdaea35d2465e01e526e8b4014828f984"}
Jul 28 17:58:11.816167 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.815984    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:58:11.817094 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.817024    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79fdg" event={"ID":"4f4bb9ff-f8d8-445e-b551-eb10ad499789","Type":"ContainerStarted","Data":"4546a2f0351d9ca100b6598debd0063380e3274e1b2873f3ac7e7eee0a43d344"}
Jul 28 17:58:11.840326 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.839843    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk" podStartSLOduration=20.748457461 podStartE2EDuration="23.839819954s" totalImagesPullingTime="3.091362493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:11.839368625 +0000 UTC m=+65.984921600" watchObservedRunningTime="2026-07-28 17:58:11.839819954 +0000 UTC m=+65.985372930"
Jul 28 17:58:11.943842 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:11.943725    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-fbhfk"
Jul 28 17:58:12.094749 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.091173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:58:12.095240 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.095023    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:58:12.106952 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.106487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f6924da-d133-4898-8aa4-0a1aa892ff2e-metrics-certs\") pod \"network-metrics-daemon-jck5m\" (UID: \"5f6924da-d133-4898-8aa4-0a1aa892ff2e\") " pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:58:12.121176 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.120935    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pbl4v\""
Jul 28 17:58:12.129435 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.129395    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jck5m"
Jul 28 17:58:12.192571 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.191960    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:58:12.192571 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.192031    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:58:12.195334 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.194845    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:58:12.195334 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.195156    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:58:12.205755 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.205691    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55e27a0d-949a-44da-9368-a62947420498-original-pull-secret\") pod \"global-pull-secret-syncer-vdwk2\" (UID: \"55e27a0d-949a-44da-9368-a62947420498\") " pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:58:12.208421 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.208262    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:58:12.216107 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.216079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbwr8\" (UniqueName: \"kubernetes.io/projected/724f5f04-2dc2-4aca-8088-812f03ebf9f8-kube-api-access-gbwr8\") pod \"network-check-target-q8q24\" (UID: \"724f5f04-2dc2-4aca-8088-812f03ebf9f8\") " pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:58:12.290926 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.290888    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jck5m"]
Jul 28 17:58:12.294868 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:12.294829    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f6924da_d133_4898_8aa4_0a1aa892ff2e.slice/crio-4f3af60ee941f5790ca1840efd7c0414f94ce9bcf389ffb0b34cfbc99b398922 WatchSource:0}: Error finding container 4f3af60ee941f5790ca1840efd7c0414f94ce9bcf389ffb0b34cfbc99b398922: Status 404 returned error can't find the container with id 4f3af60ee941f5790ca1840efd7c0414f94ce9bcf389ffb0b34cfbc99b398922
Jul 28 17:58:12.408711 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.408633    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgwbs\""
Jul 28 17:58:12.412829 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.412803    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdwk2"
Jul 28 17:58:12.416684 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.416659    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:58:12.567768 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.567732    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vdwk2"]
Jul 28 17:58:12.571558 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:12.571527    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e27a0d_949a_44da_9368_a62947420498.slice/crio-7fa65b81f6189498a4ba5232e7ae1cdc2ea33c5d4f6828317ad3a54daff51f0d WatchSource:0}: Error finding container 7fa65b81f6189498a4ba5232e7ae1cdc2ea33c5d4f6828317ad3a54daff51f0d: Status 404 returned error can't find the container with id 7fa65b81f6189498a4ba5232e7ae1cdc2ea33c5d4f6828317ad3a54daff51f0d
Jul 28 17:58:12.585737 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.585707    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q8q24"]
Jul 28 17:58:12.590044 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:12.590012    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724f5f04_2dc2_4aca_8088_812f03ebf9f8.slice/crio-8397754d489a73d559c6cfe9c59712c74ccf48a8b1103ba0610963b68f1a8544 WatchSource:0}: Error finding container 8397754d489a73d559c6cfe9c59712c74ccf48a8b1103ba0610963b68f1a8544: Status 404 returned error can't find the container with id 8397754d489a73d559c6cfe9c59712c74ccf48a8b1103ba0610963b68f1a8544
Jul 28 17:58:12.823458 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.823381    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vdwk2" event={"ID":"55e27a0d-949a-44da-9368-a62947420498","Type":"ContainerStarted","Data":"7fa65b81f6189498a4ba5232e7ae1cdc2ea33c5d4f6828317ad3a54daff51f0d"}
Jul 28 17:58:12.824871 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.824797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q8q24" event={"ID":"724f5f04-2dc2-4aca-8088-812f03ebf9f8","Type":"ContainerStarted","Data":"8397754d489a73d559c6cfe9c59712c74ccf48a8b1103ba0610963b68f1a8544"}
Jul 28 17:58:12.826366 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:12.826338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jck5m" event={"ID":"5f6924da-d133-4898-8aa4-0a1aa892ff2e","Type":"ContainerStarted","Data":"4f3af60ee941f5790ca1840efd7c0414f94ce9bcf389ffb0b34cfbc99b398922"}
Jul 28 17:58:13.836717 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:13.836659    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jck5m" event={"ID":"5f6924da-d133-4898-8aa4-0a1aa892ff2e","Type":"ContainerStarted","Data":"498d8e40f70e5e562bbc5ff48bae6ae8e8fce229b4c7a35ca8f43defd231d22d"}
Jul 28 17:58:13.837197 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:13.836725    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jck5m" event={"ID":"5f6924da-d133-4898-8aa4-0a1aa892ff2e","Type":"ContainerStarted","Data":"7bd938b49753cc7e64543cd6f37e0539fc23e0b77c14eecf254c482ae56e40cd"}
Jul 28 17:58:13.853763 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:13.853704    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jck5m" podStartSLOduration=65.741967595 podStartE2EDuration="1m6.853681811s" totalImagesPullingTime="1.111714216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:13.853103852 +0000 UTC m=+67.998656827" watchObservedRunningTime="2026-07-28 17:58:13.853681811 +0000 UTC m=+67.999234779"
Jul 28 17:58:14.109535 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:14.109496    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:58:14.113500 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:14.113468    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/976d2309-867a-4a03-a1c1-fc52341a4c05-metrics-tls\") pod \"dns-default-jd452\" (UID: \"976d2309-867a-4a03-a1c1-fc52341a4c05\") " pod="openshift-dns/dns-default-jd452"
Jul 28 17:58:14.347790 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:14.347750    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5mqwj\""
Jul 28 17:58:14.356090 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:14.356028    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jd452"
Jul 28 17:58:18.504674 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.504618    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"]
Jul 28 17:58:18.507399 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.507374    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.520982 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.520949    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 17:58:18.522974 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.522953    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 17:58:18.523335 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.523270    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 17:58:18.523335 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.523294    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 17:58:18.523471 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.523295    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 17:58:18.525003 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.524984    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 17:58:18.528765 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.528746    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 17:58:18.562432 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.562396    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"]
Jul 28 17:58:18.646279 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.646245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a814970-d088-4d63-be1d-5b168484f8ea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.646472 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.646289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ppv2\" (UniqueName: \"kubernetes.io/projected/7a814970-d088-4d63-be1d-5b168484f8ea-kube-api-access-8ppv2\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.646472 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.646337    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-ca\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.646472 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.646386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.646472 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.646424    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-hub\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.646472 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.646463    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.747534 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.747499    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a814970-d088-4d63-be1d-5b168484f8ea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.747739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.747546    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ppv2\" (UniqueName: \"kubernetes.io/projected/7a814970-d088-4d63-be1d-5b168484f8ea-kube-api-access-8ppv2\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.747739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.747581    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-ca\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.747739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.747599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.747739 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.747710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-hub\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.747931 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.747766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.748405 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.748356    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a814970-d088-4d63-be1d-5b168484f8ea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.750713 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.750688    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-ca\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.750841 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.750800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.750938 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.750910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.751458 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.751415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7a814970-d088-4d63-be1d-5b168484f8ea-hub\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.774783 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.774702    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ppv2\" (UniqueName: \"kubernetes.io/projected/7a814970-d088-4d63-be1d-5b168484f8ea-kube-api-access-8ppv2\") pod \"cluster-proxy-proxy-agent-f4b7f965-zgjph\" (UID: \"7a814970-d088-4d63-be1d-5b168484f8ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:18.825311 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:18.825272    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"
Jul 28 17:58:19.789664 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.789636    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jd452"]
Jul 28 17:58:19.793958 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:19.793755    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976d2309_867a_4a03_a1c1_fc52341a4c05.slice/crio-58fae2401c2603cf9679ff637589534730ab03ff65e127857907ba30b03dd59c WatchSource:0}: Error finding container 58fae2401c2603cf9679ff637589534730ab03ff65e127857907ba30b03dd59c: Status 404 returned error can't find the container with id 58fae2401c2603cf9679ff637589534730ab03ff65e127857907ba30b03dd59c
Jul 28 17:58:19.813794 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.813718    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph"]
Jul 28 17:58:19.819549 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:19.819515    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a814970_d088_4d63_be1d_5b168484f8ea.slice/crio-8380962db9517bf77fc76a3554cee34ead9db38bf0a1f1e650bbcd657d0422e4 WatchSource:0}: Error finding container 8380962db9517bf77fc76a3554cee34ead9db38bf0a1f1e650bbcd657d0422e4: Status 404 returned error can't find the container with id 8380962db9517bf77fc76a3554cee34ead9db38bf0a1f1e650bbcd657d0422e4
Jul 28 17:58:19.861078 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.861033    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vdwk2" event={"ID":"55e27a0d-949a-44da-9368-a62947420498","Type":"ContainerStarted","Data":"0153447b1cbdc3a648dc20259abeb40447f0383017c950d60b703191a6cb3d8c"}
Jul 28 17:58:19.862758 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.862726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79fdg" event={"ID":"4f4bb9ff-f8d8-445e-b551-eb10ad499789","Type":"ContainerStarted","Data":"1d973524505e3a844ceedf19148386c873295876958013cb12644a9f5fa826c8"}
Jul 28 17:58:19.864401 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.864373    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q8q24" event={"ID":"724f5f04-2dc2-4aca-8088-812f03ebf9f8","Type":"ContainerStarted","Data":"7679c7007f2199164d49dcfdc1551c01ba8a435efe0db2f714251032a88d0f79"}
Jul 28 17:58:19.864559 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.864425    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:58:19.865636 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.865611    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph" event={"ID":"7a814970-d088-4d63-be1d-5b168484f8ea","Type":"ContainerStarted","Data":"8380962db9517bf77fc76a3554cee34ead9db38bf0a1f1e650bbcd657d0422e4"}
Jul 28 17:58:19.866787 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.866765    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jd452" event={"ID":"976d2309-867a-4a03-a1c1-fc52341a4c05","Type":"ContainerStarted","Data":"58fae2401c2603cf9679ff637589534730ab03ff65e127857907ba30b03dd59c"}
Jul 28 17:58:19.876607 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.876400    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vdwk2" podStartSLOduration=65.808707901 podStartE2EDuration="1m12.876378567s" totalImagesPullingTime="7.067670666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:19.875999049 +0000 UTC m=+74.021552022" watchObservedRunningTime="2026-07-28 17:58:19.876378567 +0000 UTC m=+74.021931542"
Jul 28 17:58:19.894405 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:19.894343    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q8q24" podStartSLOduration=65.86584773 podStartE2EDuration="1m12.894320931s" totalImagesPullingTime="7.028473201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:19.893887437 +0000 UTC m=+74.039440406" watchObservedRunningTime="2026-07-28 17:58:19.894320931 +0000 UTC m=+74.039873906"
Jul 28 17:58:20.563495 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.563447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:20.565088 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.564494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7e625-fd4c-4c3a-8c40-798c157f5173-service-ca-bundle\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:20.565347 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.565240    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:20.570689 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.570632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e7e625-fd4c-4c3a-8c40-798c157f5173-metrics-certs\") pod \"router-default-75c4fdbf55-r9lmg\" (UID: \"c1e7e625-fd4c-4c3a-8c40-798c157f5173\") " pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:20.666466 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.665928    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:58:20.673139 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.673075    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72a6a8a7-dc6e-4b60-b06f-6f6c3497307d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pldz2\" (UID: \"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:58:20.720044 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.719765    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2vjht\""
Jul 28 17:58:20.728429 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.728123    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:20.890905 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.890868    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9g2gw\""
Jul 28 17:58:20.899440 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.899184    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"
Jul 28 17:58:20.920656 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:20.920620    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75c4fdbf55-r9lmg"]
Jul 28 17:58:20.926902 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:20.926661    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e7e625_fd4c_4c3a_8c40_798c157f5173.slice/crio-31ae1767b970f9567a9c11140cd3f55b6be1b8a2fd66b9e19d3093540611f30b WatchSource:0}: Error finding container 31ae1767b970f9567a9c11140cd3f55b6be1b8a2fd66b9e19d3093540611f30b: Status 404 returned error can't find the container with id 31ae1767b970f9567a9c11140cd3f55b6be1b8a2fd66b9e19d3093540611f30b
Jul 28 17:58:21.090744 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:21.090707    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pldz2"]
Jul 28 17:58:21.094029 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:21.093992    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a6a8a7_dc6e_4b60_b06f_6f6c3497307d.slice/crio-a59215a88507ea707f6879a0be18bf674c47ddac421be55ee5225f306cb06d84 WatchSource:0}: Error finding container a59215a88507ea707f6879a0be18bf674c47ddac421be55ee5225f306cb06d84: Status 404 returned error can't find the container with id a59215a88507ea707f6879a0be18bf674c47ddac421be55ee5225f306cb06d84
Jul 28 17:58:21.879670 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:21.879197    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg" event={"ID":"c1e7e625-fd4c-4c3a-8c40-798c157f5173","Type":"ContainerStarted","Data":"2c8b1bbe14272e2a843eaf3e3725d6265a576f6855b90c6fc855ee1fa069fc98"}
Jul 28 17:58:21.879670 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:21.879261    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg" event={"ID":"c1e7e625-fd4c-4c3a-8c40-798c157f5173","Type":"ContainerStarted","Data":"31ae1767b970f9567a9c11140cd3f55b6be1b8a2fd66b9e19d3093540611f30b"}
Jul 28 17:58:21.880688 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:21.880630    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2" event={"ID":"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d","Type":"ContainerStarted","Data":"a59215a88507ea707f6879a0be18bf674c47ddac421be55ee5225f306cb06d84"}
Jul 28 17:58:21.902217 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:21.901401    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg" podStartSLOduration=33.90137672 podStartE2EDuration="33.90137672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:21.900318082 +0000 UTC m=+76.045871057" watchObservedRunningTime="2026-07-28 17:58:21.90137672 +0000 UTC m=+76.046929695"
Jul 28 17:58:22.729486 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:22.729439    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:22.732584 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:22.732558    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:22.884593 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:22.884564    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:22.886070 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:22.886044    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-75c4fdbf55-r9lmg"
Jul 28 17:58:26.901174 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.901130    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2" event={"ID":"72a6a8a7-dc6e-4b60-b06f-6f6c3497307d","Type":"ContainerStarted","Data":"db2b6c0c8696beff0daa01cd2afe7b3dda82e3442fcf2f649cedc086e5d30852"}
Jul 28 17:58:26.903218 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.903186    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79fdg" event={"ID":"4f4bb9ff-f8d8-445e-b551-eb10ad499789","Type":"ContainerStarted","Data":"40070fd4612cde3b8a40f60adf31b02296640535514bb1c8e45fd01ac33e6998"}
Jul 28 17:58:26.905734 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.905494    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph" event={"ID":"7a814970-d088-4d63-be1d-5b168484f8ea","Type":"ContainerStarted","Data":"951585ab882ac14b4294eaab91fb3bd16e40ececd13898ba627c72e38c65f808"}
Jul 28 17:58:26.905734 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.905546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph" event={"ID":"7a814970-d088-4d63-be1d-5b168484f8ea","Type":"ContainerStarted","Data":"93b4c671932f88fbfdac9d2087591ac5f99d9458823704d1d86578646d732274"}
Jul 28 17:58:26.905734 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.905559    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph" event={"ID":"7a814970-d088-4d63-be1d-5b168484f8ea","Type":"ContainerStarted","Data":"37d1fa09223cc8a7ebfe46b27a986a2fa4463a00819a6f378a542100528cad90"}
Jul 28 17:58:26.912677 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.912637    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jd452" event={"ID":"976d2309-867a-4a03-a1c1-fc52341a4c05","Type":"ContainerStarted","Data":"2b4a32587680e5643750bdd4b911bb10fd7f2b1f27d3c37c4e867ca1976c1c52"}
Jul 28 17:58:26.912873 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.912735    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jd452" event={"ID":"976d2309-867a-4a03-a1c1-fc52341a4c05","Type":"ContainerStarted","Data":"99ce151f5e5b36b12f38d83892aca915d4c85a460a17b3d731c8ea06b6327dfe"}
Jul 28 17:58:26.912873 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.912802    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jd452"
Jul 28 17:58:26.917532 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.917474    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pldz2" podStartSLOduration=33.762333542 podStartE2EDuration="38.917453671s" totalImagesPullingTime="5.155120129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:26.916899804 +0000 UTC m=+81.062452776" watchObservedRunningTime="2026-07-28 17:58:26.917453671 +0000 UTC m=+81.063006644"
Jul 28 17:58:26.946503 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.946438    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jd452" podStartSLOduration=38.524646491 podStartE2EDuration="44.946413393s" totalImagesPullingTime="6.421766902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:26.94528961 +0000 UTC m=+81.090842585" watchObservedRunningTime="2026-07-28 17:58:26.946413393 +0000 UTC m=+81.091966367"
Jul 28 17:58:26.971327 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:26.971247    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f4b7f965-zgjph" podStartSLOduration=2.5430162039999997 podStartE2EDuration="8.97122539s" totalImagesPullingTime="6.428209186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:26.969388706 +0000 UTC m=+81.114941680" watchObservedRunningTime="2026-07-28 17:58:26.97122539 +0000 UTC m=+81.116778365"
Jul 28 17:58:29.925904 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:29.925867    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-79fdg" event={"ID":"4f4bb9ff-f8d8-445e-b551-eb10ad499789","Type":"ContainerStarted","Data":"88107be0da38f81b676d2037afbbc652dc2efdcf04a50f4555239505a39ec7d4"}
Jul 28 17:58:29.949117 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:29.949052    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-79fdg" podStartSLOduration=16.619829225 podStartE2EDuration="34.949033124s" totalImagesPullingTime="18.329203899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:29.947294982 +0000 UTC m=+84.092847955" watchObservedRunningTime="2026-07-28 17:58:29.949033124 +0000 UTC m=+84.094586098"
Jul 28 17:58:36.919888 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:36.919839    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jd452"
Jul 28 17:58:43.456250 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.456216    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b872x"]
Jul 28 17:58:43.460461 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.460444    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.470140 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.470117    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 17:58:43.470260 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.470246    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 17:58:43.470313 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.470246    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 17:58:43.470348 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.470246    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 17:58:43.470348 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.470333    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 17:58:43.471165 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.471148    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j6s8v\""
Jul 28 17:58:43.471233 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.471216    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 17:58:43.644632 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644600    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-root\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.644822 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl2nc\" (UniqueName: \"kubernetes.io/projected/87800744-9a53-4f1f-b0ea-4a220337d4ef-kube-api-access-jl2nc\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.644822 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-wtmp\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.644822 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644684    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.644822 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644743    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-textfile\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.645032 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644823    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.645032 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87800744-9a53-4f1f-b0ea-4a220337d4ef-metrics-client-ca\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.645032 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-tls\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.645032 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.644922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-sys\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.745627 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-root\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.745797 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745644    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl2nc\" (UniqueName: \"kubernetes.io/projected/87800744-9a53-4f1f-b0ea-4a220337d4ef-kube-api-access-jl2nc\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.745797 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-wtmp\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.745797 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745707    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.745797 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-root\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.745797 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745751    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-textfile\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746047 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746047 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745879    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87800744-9a53-4f1f-b0ea-4a220337d4ef-metrics-client-ca\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746047 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745890    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-wtmp\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746047 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-tls\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746047 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.745955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-sys\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746047 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:43.745992    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 17:58:43.746047 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.746050    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87800744-9a53-4f1f-b0ea-4a220337d4ef-sys\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746365 ip-10-0-132-23 kubenswrapper[2575]: E0728 17:58:43.746063    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-tls podName:87800744-9a53-4f1f-b0ea-4a220337d4ef nodeName:}" failed. No retries permitted until 2026-07-28 17:58:44.246040388 +0000 UTC m=+98.391593352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-tls") pod "node-exporter-b872x" (UID: "87800744-9a53-4f1f-b0ea-4a220337d4ef") : secret "node-exporter-tls" not found
Jul 28 17:58:43.746365 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.746112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-textfile\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746534 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.746517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.746623 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.746603    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87800744-9a53-4f1f-b0ea-4a220337d4ef-metrics-client-ca\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.748650 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.748631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:43.767235 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:43.767199    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl2nc\" (UniqueName: \"kubernetes.io/projected/87800744-9a53-4f1f-b0ea-4a220337d4ef-kube-api-access-jl2nc\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:44.249907 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:44.249849    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-tls\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:44.252115 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:44.252088    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/87800744-9a53-4f1f-b0ea-4a220337d4ef-node-exporter-tls\") pod \"node-exporter-b872x\" (UID: \"87800744-9a53-4f1f-b0ea-4a220337d4ef\") " pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:44.369588 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:44.369553    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b872x"
Jul 28 17:58:44.408458 ip-10-0-132-23 kubenswrapper[2575]: W0728 17:58:44.408425    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87800744_9a53_4f1f_b0ea_4a220337d4ef.slice/crio-e0f02c7e70c7dbbb6201f2a343ebd213ee99d2095255eff1cad0ede0c57c42b1 WatchSource:0}: Error finding container e0f02c7e70c7dbbb6201f2a343ebd213ee99d2095255eff1cad0ede0c57c42b1: Status 404 returned error can't find the container with id e0f02c7e70c7dbbb6201f2a343ebd213ee99d2095255eff1cad0ede0c57c42b1
Jul 28 17:58:44.981869 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:44.981828    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b872x" event={"ID":"87800744-9a53-4f1f-b0ea-4a220337d4ef","Type":"ContainerStarted","Data":"e0f02c7e70c7dbbb6201f2a343ebd213ee99d2095255eff1cad0ede0c57c42b1"}
Jul 28 17:58:45.986523 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:45.986432    2575 generic.go:356] "Generic (PLEG): container finished" podID="87800744-9a53-4f1f-b0ea-4a220337d4ef" containerID="8376c32070e28ec13619129286abd55f9f06b044d21871136ba7088167f8814b" exitCode=0
Jul 28 17:58:45.986966 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:45.986523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b872x" event={"ID":"87800744-9a53-4f1f-b0ea-4a220337d4ef","Type":"ContainerDied","Data":"8376c32070e28ec13619129286abd55f9f06b044d21871136ba7088167f8814b"}
Jul 28 17:58:46.992557 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:46.992520    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b872x" event={"ID":"87800744-9a53-4f1f-b0ea-4a220337d4ef","Type":"ContainerStarted","Data":"ae364d9b875da740ccb634460080a916013c07c10dfc8e87391e9b93a3a61119"}
Jul 28 17:58:46.992986 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:46.992586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b872x" event={"ID":"87800744-9a53-4f1f-b0ea-4a220337d4ef","Type":"ContainerStarted","Data":"68da3f533b8e5b8b6c504660ac04a5459a49e297ec6f41e9d568e6a78374a468"}
Jul 28 17:58:47.033386 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:47.033325    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b872x" podStartSLOduration=2.839375265 podStartE2EDuration="4.033302314s" totalImagesPullingTime="1.133267001s" totalInitContainerRuntime="60.660048ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:47.031812712 +0000 UTC m=+101.177365685" watchObservedRunningTime="2026-07-28 17:58:47.033302314 +0000 UTC m=+101.178855290"
Jul 28 17:58:50.875657 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:58:50.875626    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q8q24"
Jul 28 17:59:15.170782 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:59:15.170754    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75c4fdbf55-r9lmg_c1e7e625-fd4c-4c3a-8c40-798c157f5173/router/0.log"
Jul 28 17:59:15.215683 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:59:15.215646    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tsm5b_4d687f14-4aab-4083-b44d-1ed34c9a051c/serve-healthcheck-canary/0.log"
Jul 28 17:59:24.116186 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:59:24.116148    2575 generic.go:356] "Generic (PLEG): container finished" podID="f0fd56f1-1ed2-4ffc-b730-b575cfe7380e" containerID="6fea10078bb0a06cee3977ca7f386fcd337ac634c7919a895af3b47bb8baa40e" exitCode=0
Jul 28 17:59:24.116633 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:59:24.116216    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc" event={"ID":"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e","Type":"ContainerDied","Data":"6fea10078bb0a06cee3977ca7f386fcd337ac634c7919a895af3b47bb8baa40e"}
Jul 28 17:59:24.116633 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:59:24.116496    2575 scope.go:117] "RemoveContainer" containerID="6fea10078bb0a06cee3977ca7f386fcd337ac634c7919a895af3b47bb8baa40e"
Jul 28 17:59:25.125573 ip-10-0-132-23 kubenswrapper[2575]: I0728 17:59:25.125540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc" event={"ID":"f0fd56f1-1ed2-4ffc-b730-b575cfe7380e","Type":"ContainerStarted","Data":"9498cc7f3af1202b042fca8215c4de198626c8dcd5558c88e9b427ad85ffe386"}
Jul 28 18:02:06.257085 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:02:06.257055    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:02:06.262735 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:02:06.262705    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:02:06.266364 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:02:06.266345    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 18:04:36.817457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.817372    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dz7gz"]
Jul 28 18:04:36.819488 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.819469    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:36.823374 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.822536    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 18:04:36.824694 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.824667    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 18:04:36.824991 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.824966    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jzh6j\""
Jul 28 18:04:36.831424 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.831393    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dz7gz"]
Jul 28 18:04:36.894528 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.894489    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km4bm\" (UniqueName: \"kubernetes.io/projected/0b09a083-c602-49d0-9667-4e1ecf1ee8c4-kube-api-access-km4bm\") pod \"cert-manager-55b4d954d5-dz7gz\" (UID: \"0b09a083-c602-49d0-9667-4e1ecf1ee8c4\") " pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:36.894528 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.894530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b09a083-c602-49d0-9667-4e1ecf1ee8c4-bound-sa-token\") pod \"cert-manager-55b4d954d5-dz7gz\" (UID: \"0b09a083-c602-49d0-9667-4e1ecf1ee8c4\") " pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:36.995189 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.995149    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km4bm\" (UniqueName: \"kubernetes.io/projected/0b09a083-c602-49d0-9667-4e1ecf1ee8c4-kube-api-access-km4bm\") pod \"cert-manager-55b4d954d5-dz7gz\" (UID: \"0b09a083-c602-49d0-9667-4e1ecf1ee8c4\") " pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:36.995396 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:36.995197    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b09a083-c602-49d0-9667-4e1ecf1ee8c4-bound-sa-token\") pod \"cert-manager-55b4d954d5-dz7gz\" (UID: \"0b09a083-c602-49d0-9667-4e1ecf1ee8c4\") " pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:37.005526 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:37.005496    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b09a083-c602-49d0-9667-4e1ecf1ee8c4-bound-sa-token\") pod \"cert-manager-55b4d954d5-dz7gz\" (UID: \"0b09a083-c602-49d0-9667-4e1ecf1ee8c4\") " pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:37.005712 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:37.005563    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km4bm\" (UniqueName: \"kubernetes.io/projected/0b09a083-c602-49d0-9667-4e1ecf1ee8c4-kube-api-access-km4bm\") pod \"cert-manager-55b4d954d5-dz7gz\" (UID: \"0b09a083-c602-49d0-9667-4e1ecf1ee8c4\") " pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:37.131686 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:37.131594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dz7gz"
Jul 28 18:04:37.268141 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:37.268108    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dz7gz"]
Jul 28 18:04:37.271512 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:04:37.271484    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b09a083_c602_49d0_9667_4e1ecf1ee8c4.slice/crio-530ba3d5a6b7c32e122b6620b10185b12e81cfed895499f86804fdcfd0beb7c8 WatchSource:0}: Error finding container 530ba3d5a6b7c32e122b6620b10185b12e81cfed895499f86804fdcfd0beb7c8: Status 404 returned error can't find the container with id 530ba3d5a6b7c32e122b6620b10185b12e81cfed895499f86804fdcfd0beb7c8
Jul 28 18:04:37.273465 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:37.273446    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:04:38.202900 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:38.202830    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dz7gz" event={"ID":"0b09a083-c602-49d0-9667-4e1ecf1ee8c4","Type":"ContainerStarted","Data":"530ba3d5a6b7c32e122b6620b10185b12e81cfed895499f86804fdcfd0beb7c8"}
Jul 28 18:04:41.216186 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:41.216153    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dz7gz" event={"ID":"0b09a083-c602-49d0-9667-4e1ecf1ee8c4","Type":"ContainerStarted","Data":"68777e2dcb9d99f5eee9f3fc9d33dbd82e616f66e80a234e7312364f949d2687"}
Jul 28 18:04:41.232100 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:41.232046    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-dz7gz" podStartSLOduration=2.295077699 podStartE2EDuration="5.232027411s" totalImagesPullingTime="2.936949712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:04:41.230936098 +0000 UTC m=+455.376489169" watchObservedRunningTime="2026-07-28 18:04:41.232027411 +0000 UTC m=+455.377580385"
Jul 28 18:04:53.024435 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.024397    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"]
Jul 28 18:04:53.028541 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.028519    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.032284 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.032255    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 18:04:53.032431 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.032287    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 18:04:53.032431 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.032312    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 18:04:53.032431 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.032385    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 18:04:53.032431 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.032417    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f6wlw\""
Jul 28 18:04:53.055337 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.055306    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"]
Jul 28 18:04:53.126679 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.126643    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jvt6\" (UniqueName: \"kubernetes.io/projected/c6a199ef-cf2c-4bf4-876e-3936961e980c-kube-api-access-8jvt6\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.126896 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.126703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6a199ef-cf2c-4bf4-876e-3936961e980c-manifests\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.126896 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.126733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6a199ef-cf2c-4bf4-876e-3936961e980c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.126896 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.126759    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6a199ef-cf2c-4bf4-876e-3936961e980c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.126896 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.126779    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6a199ef-cf2c-4bf4-876e-3936961e980c-tmp\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.227702 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.227659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6a199ef-cf2c-4bf4-876e-3936961e980c-manifests\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.227702 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.227697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6a199ef-cf2c-4bf4-876e-3936961e980c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.227959 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.227722    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6a199ef-cf2c-4bf4-876e-3936961e980c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.227959 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.227837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6a199ef-cf2c-4bf4-876e-3936961e980c-tmp\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.227959 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.227905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jvt6\" (UniqueName: \"kubernetes.io/projected/c6a199ef-cf2c-4bf4-876e-3936961e980c-kube-api-access-8jvt6\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.228110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.228093    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6a199ef-cf2c-4bf4-876e-3936961e980c-manifests\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.230114 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.230089    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6a199ef-cf2c-4bf4-876e-3936961e980c-tmp\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.230330 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.230312    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6a199ef-cf2c-4bf4-876e-3936961e980c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.230456 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.230434    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6a199ef-cf2c-4bf4-876e-3936961e980c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.237791 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.237763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jvt6\" (UniqueName: \"kubernetes.io/projected/c6a199ef-cf2c-4bf4-876e-3936961e980c-kube-api-access-8jvt6\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9\" (UID: \"c6a199ef-cf2c-4bf4-876e-3936961e980c\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.338713 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.338624    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:53.482206 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:53.482183    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"]
Jul 28 18:04:53.484872 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:04:53.484833    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a199ef_cf2c_4bf4_876e_3936961e980c.slice/crio-db3acd068f023a59581ec881c317de4b51741423fededb7ccdae4d085ef6fdf2 WatchSource:0}: Error finding container db3acd068f023a59581ec881c317de4b51741423fededb7ccdae4d085ef6fdf2: Status 404 returned error can't find the container with id db3acd068f023a59581ec881c317de4b51741423fededb7ccdae4d085ef6fdf2
Jul 28 18:04:54.262937 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:54.262844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9" event={"ID":"c6a199ef-cf2c-4bf4-876e-3936961e980c","Type":"ContainerStarted","Data":"db3acd068f023a59581ec881c317de4b51741423fededb7ccdae4d085ef6fdf2"}
Jul 28 18:04:57.276533 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:57.276488    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9" event={"ID":"c6a199ef-cf2c-4bf4-876e-3936961e980c","Type":"ContainerStarted","Data":"dc0a39fa7122973e1faabfdeb5b81b53f850886fcbd51db803a79543f88a00bc"}
Jul 28 18:04:58.281673 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:58.281633    2575 generic.go:356] "Generic (PLEG): container finished" podID="c6a199ef-cf2c-4bf4-876e-3936961e980c" containerID="dc0a39fa7122973e1faabfdeb5b81b53f850886fcbd51db803a79543f88a00bc" exitCode=0
Jul 28 18:04:58.282095 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:58.281722    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9" event={"ID":"c6a199ef-cf2c-4bf4-876e-3936961e980c","Type":"ContainerDied","Data":"dc0a39fa7122973e1faabfdeb5b81b53f850886fcbd51db803a79543f88a00bc"}
Jul 28 18:04:59.287456 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:59.287415    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9" event={"ID":"c6a199ef-cf2c-4bf4-876e-3936961e980c","Type":"ContainerStarted","Data":"b70d84e1b073bfc0905952a27f024f3b7edfbd5749f140d14903f3e966bbbf31"}
Jul 28 18:04:59.287878 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:59.287536    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:04:59.311736 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:04:59.311676    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9" podStartSLOduration=2.762685027 podStartE2EDuration="7.311654346s" totalImagesPullingTime="3.472049048s" totalInitContainerRuntime="1.076920271s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:04:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:04:59.311121295 +0000 UTC m=+473.456674264" watchObservedRunningTime="2026-07-28 18:04:59.311654346 +0000 UTC m=+473.457207320"
Jul 28 18:05:10.293931 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:10.293900    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9"
Jul 28 18:05:13.782023 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.781989    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"]
Jul 28 18:05:13.789748 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.789725    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:13.819286 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.819253    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"]
Jul 28 18:05:13.827764 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.827736    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 18:05:13.828175 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.828153    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 18:05:13.896010 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.895969    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tls-certs\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:13.896225 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.896023    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zt2t\" (UniqueName: \"kubernetes.io/projected/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-kube-api-access-8zt2t\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:13.896225 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.896106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tmp\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:13.997399 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.997362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zt2t\" (UniqueName: \"kubernetes.io/projected/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-kube-api-access-8zt2t\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:13.997606 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.997411    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tmp\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:13.997606 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.997461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tls-certs\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:13.997606 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:05:13.997573    2575 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 18:05:13.997749 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:05:13.997653    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tls-certs podName:87c2c23e-ef58-4857-b2f1-85e0f5d667c8 nodeName:}" failed. No retries permitted until 2026-07-28 18:05:14.497631696 +0000 UTC m=+488.643184661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tls-certs") pod "kube-auth-proxy-7cfd4f6bf9-pttnj" (UID: "87c2c23e-ef58-4857-b2f1-85e0f5d667c8") : secret "kube-auth-proxy-tls" not found
Jul 28 18:05:13.999754 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:13.999731    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tmp\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:14.035738 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.035658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zt2t\" (UniqueName: \"kubernetes.io/projected/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-kube-api-access-8zt2t\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:14.430662 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.430570    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-t5252"]
Jul 28 18:05:14.433112 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.433086    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:14.438834 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:05:14.438796    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"odh-model-controller-7998c67dc4-t5252\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="508100a5-0b7d-41a5-b761-2b4d53f14d9e" pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:14.470435 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.470402    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 18:05:14.470759 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.470742    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rkbkz\""
Jul 28 18:05:14.501157 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.501125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tls-certs\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:14.503642 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.503605    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c2c23e-ef58-4857-b2f1-85e0f5d667c8-tls-certs\") pod \"kube-auth-proxy-7cfd4f6bf9-pttnj\" (UID: \"87c2c23e-ef58-4857-b2f1-85e0f5d667c8\") " pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:14.510937 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.510901    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-t5252"]
Jul 28 18:05:14.602540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.602501    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/508100a5-0b7d-41a5-b761-2b4d53f14d9e-cert\") pod \"odh-model-controller-7998c67dc4-t5252\" (UID: \"508100a5-0b7d-41a5-b761-2b4d53f14d9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:14.602712 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.602547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl97c\" (UniqueName: \"kubernetes.io/projected/508100a5-0b7d-41a5-b761-2b4d53f14d9e-kube-api-access-fl97c\") pod \"odh-model-controller-7998c67dc4-t5252\" (UID: \"508100a5-0b7d-41a5-b761-2b4d53f14d9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:14.698584 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.698496    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"
Jul 28 18:05:14.703588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.703565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/508100a5-0b7d-41a5-b761-2b4d53f14d9e-cert\") pod \"odh-model-controller-7998c67dc4-t5252\" (UID: \"508100a5-0b7d-41a5-b761-2b4d53f14d9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:14.703669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.703604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl97c\" (UniqueName: \"kubernetes.io/projected/508100a5-0b7d-41a5-b761-2b4d53f14d9e-kube-api-access-fl97c\") pod \"odh-model-controller-7998c67dc4-t5252\" (UID: \"508100a5-0b7d-41a5-b761-2b4d53f14d9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:14.703743 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:05:14.703723    2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 18:05:14.703804 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:05:14.703794    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508100a5-0b7d-41a5-b761-2b4d53f14d9e-cert podName:508100a5-0b7d-41a5-b761-2b4d53f14d9e nodeName:}" failed. No retries permitted until 2026-07-28 18:05:15.203776661 +0000 UTC m=+489.349329613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/508100a5-0b7d-41a5-b761-2b4d53f14d9e-cert") pod "odh-model-controller-7998c67dc4-t5252" (UID: "508100a5-0b7d-41a5-b761-2b4d53f14d9e") : secret "odh-model-controller-webhook-cert" not found
Jul 28 18:05:14.757436 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.757402    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl97c\" (UniqueName: \"kubernetes.io/projected/508100a5-0b7d-41a5-b761-2b4d53f14d9e-kube-api-access-fl97c\") pod \"odh-model-controller-7998c67dc4-t5252\" (UID: \"508100a5-0b7d-41a5-b761-2b4d53f14d9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:14.900555 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:14.900515    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj"]
Jul 28 18:05:14.903661 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:05:14.903631    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c2c23e_ef58_4857_b2f1_85e0f5d667c8.slice/crio-0649c71c523492f6c35c1e6ea8df3e56e505bc7491821a30b79f44e2c1e132b2 WatchSource:0}: Error finding container 0649c71c523492f6c35c1e6ea8df3e56e505bc7491821a30b79f44e2c1e132b2: Status 404 returned error can't find the container with id 0649c71c523492f6c35c1e6ea8df3e56e505bc7491821a30b79f44e2c1e132b2
Jul 28 18:05:15.207546 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:15.207504    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/508100a5-0b7d-41a5-b761-2b4d53f14d9e-cert\") pod \"odh-model-controller-7998c67dc4-t5252\" (UID: \"508100a5-0b7d-41a5-b761-2b4d53f14d9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:15.210047 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:15.210021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/508100a5-0b7d-41a5-b761-2b4d53f14d9e-cert\") pod \"odh-model-controller-7998c67dc4-t5252\" (UID: \"508100a5-0b7d-41a5-b761-2b4d53f14d9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:15.343231 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:15.343189    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:15.350063 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:15.350027    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj" event={"ID":"87c2c23e-ef58-4857-b2f1-85e0f5d667c8","Type":"ContainerStarted","Data":"0649c71c523492f6c35c1e6ea8df3e56e505bc7491821a30b79f44e2c1e132b2"}
Jul 28 18:05:15.507707 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:15.507670    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-t5252"]
Jul 28 18:05:15.511321 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:05:15.511281    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508100a5_0b7d_41a5_b761_2b4d53f14d9e.slice/crio-0bac98ca905c78e3027c97265ea20c9088f4ece734d73a3cc4ce3395957038cf WatchSource:0}: Error finding container 0bac98ca905c78e3027c97265ea20c9088f4ece734d73a3cc4ce3395957038cf: Status 404 returned error can't find the container with id 0bac98ca905c78e3027c97265ea20c9088f4ece734d73a3cc4ce3395957038cf
Jul 28 18:05:16.356796 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.356758    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-t5252" event={"ID":"508100a5-0b7d-41a5-b761-2b4d53f14d9e","Type":"ContainerStarted","Data":"0bac98ca905c78e3027c97265ea20c9088f4ece734d73a3cc4ce3395957038cf"}
Jul 28 18:05:16.403992 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.403958    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7j5jt"]
Jul 28 18:05:16.407000 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.406974    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.411625 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.411595    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 18:05:16.411799 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.411690    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-27nn8\""
Jul 28 18:05:16.516524 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.516485    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9bb11c-9a86-47f7-81e5-ae63ec81dc38-tls-certs\") pod \"model-serving-api-54d89bd5fc-7j5jt\" (UID: \"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38\") " pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.516722 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.516538    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wzpm\" (UniqueName: \"kubernetes.io/projected/9f9bb11c-9a86-47f7-81e5-ae63ec81dc38-kube-api-access-6wzpm\") pod \"model-serving-api-54d89bd5fc-7j5jt\" (UID: \"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38\") " pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.523525 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.523436    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7j5jt"]
Jul 28 18:05:16.618467 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.617987    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9bb11c-9a86-47f7-81e5-ae63ec81dc38-tls-certs\") pod \"model-serving-api-54d89bd5fc-7j5jt\" (UID: \"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38\") " pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.618467 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.618036    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wzpm\" (UniqueName: \"kubernetes.io/projected/9f9bb11c-9a86-47f7-81e5-ae63ec81dc38-kube-api-access-6wzpm\") pod \"model-serving-api-54d89bd5fc-7j5jt\" (UID: \"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38\") " pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.621670 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.621642    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f9bb11c-9a86-47f7-81e5-ae63ec81dc38-tls-certs\") pod \"model-serving-api-54d89bd5fc-7j5jt\" (UID: \"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38\") " pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.656341 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.656302    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wzpm\" (UniqueName: \"kubernetes.io/projected/9f9bb11c-9a86-47f7-81e5-ae63ec81dc38-kube-api-access-6wzpm\") pod \"model-serving-api-54d89bd5fc-7j5jt\" (UID: \"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38\") " pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.723819 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.723785    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:16.928229 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:16.928140    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7j5jt"]
Jul 28 18:05:16.933175 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:05:16.932102    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9bb11c_9a86_47f7_81e5_ae63ec81dc38.slice/crio-daa04ff0858c5e3aeaed49f99a7f6fd9f9c9d934295d710625300a5d445bc52c WatchSource:0}: Error finding container daa04ff0858c5e3aeaed49f99a7f6fd9f9c9d934295d710625300a5d445bc52c: Status 404 returned error can't find the container with id daa04ff0858c5e3aeaed49f99a7f6fd9f9c9d934295d710625300a5d445bc52c
Jul 28 18:05:17.362556 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:17.362507    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt" event={"ID":"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38","Type":"ContainerStarted","Data":"daa04ff0858c5e3aeaed49f99a7f6fd9f9c9d934295d710625300a5d445bc52c"}
Jul 28 18:05:18.337595 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.337516    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"]
Jul 28 18:05:18.341131 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.341102    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.343886 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.343838    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wp8m4\""
Jul 28 18:05:18.344476 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.344444    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 18:05:18.344997 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.344964    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 18:05:18.344997 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.344966    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 18:05:18.345159 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.345033    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 18:05:18.357217 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.357138    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"]
Jul 28 18:05:18.434937 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.434900    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.435427 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.434953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.435427 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.435054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.435427 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.435140    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.435427 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.435173    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.435427 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.435203    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctbwf\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-kube-api-access-ctbwf\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.435655 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.435515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.536466 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.536423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.536668 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.536482    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.536668 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.536529    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.536668 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.536588    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.536896 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.536723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.536896 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.536775    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctbwf\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-kube-api-access-ctbwf\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.536896 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.536811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.537216 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.537189    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.539475 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.539445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.539627 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.539610    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.539693 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.539661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.539736 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.539688    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.553416 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.553381    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.555833 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.555802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctbwf\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-kube-api-access-ctbwf\") pod \"istiod-openshift-gateway-7b7995d5cd-ddkfm\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:18.654039 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:18.653939    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:19.563338 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:19.563312    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"]
Jul 28 18:05:19.565745 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:05:19.565717    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73aeb98c_a0c3_4989_9c5f_1c5a448d345a.slice/crio-79bd218c989d638b4a37b2caa125c5bf0aa4e8cc452e9a003999f14b27c0b159 WatchSource:0}: Error finding container 79bd218c989d638b4a37b2caa125c5bf0aa4e8cc452e9a003999f14b27c0b159: Status 404 returned error can't find the container with id 79bd218c989d638b4a37b2caa125c5bf0aa4e8cc452e9a003999f14b27c0b159
Jul 28 18:05:20.377387 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.377353    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-t5252" event={"ID":"508100a5-0b7d-41a5-b761-2b4d53f14d9e","Type":"ContainerStarted","Data":"3ff34e4e48d2ad9a90632dabd966e1512ff568bedff3050fa74653d55038a9a5"}
Jul 28 18:05:20.377598 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.377447    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:20.378663 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.378641    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm" event={"ID":"73aeb98c-a0c3-4989-9c5f-1c5a448d345a","Type":"ContainerStarted","Data":"79bd218c989d638b4a37b2caa125c5bf0aa4e8cc452e9a003999f14b27c0b159"}
Jul 28 18:05:20.380025 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.380002    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj" event={"ID":"87c2c23e-ef58-4857-b2f1-85e0f5d667c8","Type":"ContainerStarted","Data":"2cb4b75e1c1f34e6622df00fe35b6e0679b25006f8b761f1965cb6f85a3300e5"}
Jul 28 18:05:20.381337 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.381315    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt" event={"ID":"9f9bb11c-9a86-47f7-81e5-ae63ec81dc38","Type":"ContainerStarted","Data":"cabe1d2bb6eb6029dc558a63a4e94703b4c8d020a6d8f7e4490e5ad4e8cab5ce"}
Jul 28 18:05:20.381446 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.381434    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:20.429778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.429700    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-t5252" podStartSLOduration=2.590427763 podStartE2EDuration="6.429677552s" totalImagesPullingTime="3.839249789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:20.428590381 +0000 UTC m=+494.574143362" watchObservedRunningTime="2026-07-28 18:05:20.429677552 +0000 UTC m=+494.575230527"
Jul 28 18:05:20.501476 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:20.501419    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt" podStartSLOduration=2.093474655 podStartE2EDuration="4.501400372s" totalImagesPullingTime="2.407925717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:20.465963509 +0000 UTC m=+494.611516494" watchObservedRunningTime="2026-07-28 18:05:20.501400372 +0000 UTC m=+494.646953345"
Jul 28 18:05:22.988685 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:22.988635    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:05:22.989051 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:22.988743    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:05:23.396724 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:23.396680    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm" event={"ID":"73aeb98c-a0c3-4989-9c5f-1c5a448d345a","Type":"ContainerStarted","Data":"09c62311c0e183bb538e79e62bff72ec7ed225c27d20b39c0706b3ae28712c60"}
Jul 28 18:05:23.396950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:23.396811    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:23.471277 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:23.471225    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm" podStartSLOduration=2.050646458 podStartE2EDuration="5.471208013s" totalImagesPullingTime="3.420561555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:23.469474566 +0000 UTC m=+497.615027539" watchObservedRunningTime="2026-07-28 18:05:23.471208013 +0000 UTC m=+497.616760986"
Jul 28 18:05:23.471458 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:23.471325    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cfd4f6bf9-pttnj" podStartSLOduration=6.031142328 podStartE2EDuration="10.471320826s" totalImagesPullingTime="4.440178498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:20.501033307 +0000 UTC m=+494.646586281" watchObservedRunningTime="2026-07-28 18:05:23.471320826 +0000 UTC m=+497.616873800"
Jul 28 18:05:24.402236 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.402207    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:24.507872 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.507814    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"]
Jul 28 18:05:24.511100 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.511073    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.514933 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.514911    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ddcr2\""
Jul 28 18:05:24.574066 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.574033    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"]
Jul 28 18:05:24.592303 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592270    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592303 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592511 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592511 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592397    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592511 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592443    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592635 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592495    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592635 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592635 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.592773 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.592658    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5lxx\" (UniqueName: \"kubernetes.io/projected/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-kube-api-access-t5lxx\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.693930 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.693817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.693930 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.693880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.693930 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.693916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.693944    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.693967    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694083    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694144    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5lxx\" (UniqueName: \"kubernetes.io/projected/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-kube-api-access-t5lxx\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694527 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694527 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694385    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694527 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694422    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694686 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694612    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.694984 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.694956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.696415 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.696392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.696917 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.696894    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.730109 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.730074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.730248 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.730140    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5lxx\" (UniqueName: \"kubernetes.io/projected/29dd8780-99bf-4514-a5e0-cc99d3dd84e0-kube-api-access-t5lxx\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th\" (UID: \"29dd8780-99bf-4514-a5e0-cc99d3dd84e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:24.821068 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:24.821026    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:25.039758 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:25.039733    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"]
Jul 28 18:05:25.041610 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:05:25.041585    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29dd8780_99bf_4514_a5e0_cc99d3dd84e0.slice/crio-ce60d9f4afee919da6d07b48feaf301aa1f07130e1dee01aa3360f5a26f4febd WatchSource:0}: Error finding container ce60d9f4afee919da6d07b48feaf301aa1f07130e1dee01aa3360f5a26f4febd: Status 404 returned error can't find the container with id ce60d9f4afee919da6d07b48feaf301aa1f07130e1dee01aa3360f5a26f4febd
Jul 28 18:05:25.409766 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:25.409679    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th" event={"ID":"29dd8780-99bf-4514-a5e0-cc99d3dd84e0","Type":"ContainerStarted","Data":"ce60d9f4afee919da6d07b48feaf301aa1f07130e1dee01aa3360f5a26f4febd"}
Jul 28 18:05:27.741736 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.741699    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:05:27.742097 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.741775    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:05:27.742097 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.741804    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:05:27.889643 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.889609    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"]
Jul 28 18:05:27.893163 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.893134    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:27.901094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.901067    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-htz7p\""
Jul 28 18:05:27.901268 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.901068    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 18:05:27.920519 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.920467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdqj6\" (UniqueName: \"kubernetes.io/projected/fd0ad590-6443-4893-8bf9-6f7be58ecad8-kube-api-access-fdqj6\") pod \"kserve-controller-manager-d6f4b65b8-7qx6d\" (UID: \"fd0ad590-6443-4893-8bf9-6f7be58ecad8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:27.920519 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.920515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd0ad590-6443-4893-8bf9-6f7be58ecad8-cert\") pod \"kserve-controller-manager-d6f4b65b8-7qx6d\" (UID: \"fd0ad590-6443-4893-8bf9-6f7be58ecad8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:27.944191 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:27.944144    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"]
Jul 28 18:05:28.021745 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.021712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd0ad590-6443-4893-8bf9-6f7be58ecad8-cert\") pod \"kserve-controller-manager-d6f4b65b8-7qx6d\" (UID: \"fd0ad590-6443-4893-8bf9-6f7be58ecad8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:28.021968 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.021912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdqj6\" (UniqueName: \"kubernetes.io/projected/fd0ad590-6443-4893-8bf9-6f7be58ecad8-kube-api-access-fdqj6\") pod \"kserve-controller-manager-d6f4b65b8-7qx6d\" (UID: \"fd0ad590-6443-4893-8bf9-6f7be58ecad8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:28.024521 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.024493    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd0ad590-6443-4893-8bf9-6f7be58ecad8-cert\") pod \"kserve-controller-manager-d6f4b65b8-7qx6d\" (UID: \"fd0ad590-6443-4893-8bf9-6f7be58ecad8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:28.036238 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.036197    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"]
Jul 28 18:05:28.040346 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.040316    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.048981 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.048959    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zgdvl\""
Jul 28 18:05:28.050192 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.050169    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 18:05:28.063874 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.063820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdqj6\" (UniqueName: \"kubernetes.io/projected/fd0ad590-6443-4893-8bf9-6f7be58ecad8-kube-api-access-fdqj6\") pod \"kserve-controller-manager-d6f4b65b8-7qx6d\" (UID: \"fd0ad590-6443-4893-8bf9-6f7be58ecad8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:28.066893 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.066866    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"]
Jul 28 18:05:28.122671 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.122640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb7e6d94-5676-4c6f-b149-6173d4d563a1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b5jmj\" (UID: \"fb7e6d94-5676-4c6f-b149-6173d4d563a1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.122872 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.122692    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfqdl\" (UniqueName: \"kubernetes.io/projected/fb7e6d94-5676-4c6f-b149-6173d4d563a1-kube-api-access-mfqdl\") pod \"llmisvc-controller-manager-777c6bd7d5-b5jmj\" (UID: \"fb7e6d94-5676-4c6f-b149-6173d4d563a1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.205894 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.205833    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:28.223553 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.223515    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb7e6d94-5676-4c6f-b149-6173d4d563a1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b5jmj\" (UID: \"fb7e6d94-5676-4c6f-b149-6173d4d563a1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.223699 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.223605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfqdl\" (UniqueName: \"kubernetes.io/projected/fb7e6d94-5676-4c6f-b149-6173d4d563a1-kube-api-access-mfqdl\") pod \"llmisvc-controller-manager-777c6bd7d5-b5jmj\" (UID: \"fb7e6d94-5676-4c6f-b149-6173d4d563a1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.226111 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.226091    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb7e6d94-5676-4c6f-b149-6173d4d563a1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b5jmj\" (UID: \"fb7e6d94-5676-4c6f-b149-6173d4d563a1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.247152 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.247112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfqdl\" (UniqueName: \"kubernetes.io/projected/fb7e6d94-5676-4c6f-b149-6173d4d563a1-kube-api-access-mfqdl\") pod \"llmisvc-controller-manager-777c6bd7d5-b5jmj\" (UID: \"fb7e6d94-5676-4c6f-b149-6173d4d563a1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.350506 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.350469    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:28.378067 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.378030    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"]
Jul 28 18:05:28.381913 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:05:28.381886    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0ad590_6443_4893_8bf9_6f7be58ecad8.slice/crio-1dd78b4ae60a877da3b3dc6d2a1e223d0807da4b786c7b92803ca343b6eee12f WatchSource:0}: Error finding container 1dd78b4ae60a877da3b3dc6d2a1e223d0807da4b786c7b92803ca343b6eee12f: Status 404 returned error can't find the container with id 1dd78b4ae60a877da3b3dc6d2a1e223d0807da4b786c7b92803ca343b6eee12f
Jul 28 18:05:28.424256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.424220    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th" event={"ID":"29dd8780-99bf-4514-a5e0-cc99d3dd84e0","Type":"ContainerStarted","Data":"2c2f5bf1a1237e9774a5f2c11dab4522c171745491e84335a0c3e4ee2daeb86c"}
Jul 28 18:05:28.425413 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.425386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d" event={"ID":"fd0ad590-6443-4893-8bf9-6f7be58ecad8","Type":"ContainerStarted","Data":"1dd78b4ae60a877da3b3dc6d2a1e223d0807da4b786c7b92803ca343b6eee12f"}
Jul 28 18:05:28.477886 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.477816    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th" podStartSLOduration=1.780093471 podStartE2EDuration="4.477799348s" totalImagesPullingTime="2.697705877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:28.477090179 +0000 UTC m=+502.622643149" watchObservedRunningTime="2026-07-28 18:05:28.477799348 +0000 UTC m=+502.623352323"
Jul 28 18:05:28.502749 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.502721    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"]
Jul 28 18:05:28.504660 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:05:28.504633    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfb7e6d94_5676_4c6f_b149_6173d4d563a1.slice/crio-918a2aee34c728671e045c0ceb82cb9741c952d82ce1ff71e3ed2dee9cf16650 WatchSource:0}: Error finding container 918a2aee34c728671e045c0ceb82cb9741c952d82ce1ff71e3ed2dee9cf16650: Status 404 returned error can't find the container with id 918a2aee34c728671e045c0ceb82cb9741c952d82ce1ff71e3ed2dee9cf16650
Jul 28 18:05:28.822212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.822166    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:28.827185 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:28.827161    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:29.213830 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.213735    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"]
Jul 28 18:05:29.214162 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.214057    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm" podUID="73aeb98c-a0c3-4989-9c5f-1c5a448d345a" containerName="discovery" containerID="cri-o://09c62311c0e183bb538e79e62bff72ec7ed225c27d20b39c0706b3ae28712c60" gracePeriod=30
Jul 28 18:05:29.434765 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.434572    2575 generic.go:356] "Generic (PLEG): container finished" podID="73aeb98c-a0c3-4989-9c5f-1c5a448d345a" containerID="09c62311c0e183bb538e79e62bff72ec7ed225c27d20b39c0706b3ae28712c60" exitCode=0
Jul 28 18:05:29.434765 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.434691    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm" event={"ID":"73aeb98c-a0c3-4989-9c5f-1c5a448d345a","Type":"ContainerDied","Data":"09c62311c0e183bb538e79e62bff72ec7ed225c27d20b39c0706b3ae28712c60"}
Jul 28 18:05:29.436933 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.436598    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj" event={"ID":"fb7e6d94-5676-4c6f-b149-6173d4d563a1","Type":"ContainerStarted","Data":"918a2aee34c728671e045c0ceb82cb9741c952d82ce1ff71e3ed2dee9cf16650"}
Jul 28 18:05:29.436933 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.436795    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:29.438451 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.438231    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th"
Jul 28 18:05:29.517564 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.517533    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:29.534483 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534453    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-local-certs\") pod \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") "
Jul 28 18:05:29.534483 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534486    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-dns-cert\") pod \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") "
Jul 28 18:05:29.534719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534537    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-kubeconfig\") pod \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") "
Jul 28 18:05:29.534719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534561    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-ca-configmap\") pod \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") "
Jul 28 18:05:29.534719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534582    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-token\") pod \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") "
Jul 28 18:05:29.534719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534608    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctbwf\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-kube-api-access-ctbwf\") pod \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") "
Jul 28 18:05:29.534719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534674    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-cacerts\") pod \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\" (UID: \"73aeb98c-a0c3-4989-9c5f-1c5a448d345a\") "
Jul 28 18:05:29.535021 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.534957    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "73aeb98c-a0c3-4989-9c5f-1c5a448d345a" (UID: "73aeb98c-a0c3-4989-9c5f-1c5a448d345a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:05:29.538336 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.538297    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-kube-api-access-ctbwf" (OuterVolumeSpecName: "kube-api-access-ctbwf") pod "73aeb98c-a0c3-4989-9c5f-1c5a448d345a" (UID: "73aeb98c-a0c3-4989-9c5f-1c5a448d345a"). InnerVolumeSpecName "kube-api-access-ctbwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:05:29.539749 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.539701    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-token" (OuterVolumeSpecName: "istio-token") pod "73aeb98c-a0c3-4989-9c5f-1c5a448d345a" (UID: "73aeb98c-a0c3-4989-9c5f-1c5a448d345a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:05:29.540100 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.540071    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "73aeb98c-a0c3-4989-9c5f-1c5a448d345a" (UID: "73aeb98c-a0c3-4989-9c5f-1c5a448d345a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:05:29.541537 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.541500    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "73aeb98c-a0c3-4989-9c5f-1c5a448d345a" (UID: "73aeb98c-a0c3-4989-9c5f-1c5a448d345a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:05:29.541890 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.541840    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-cacerts" (OuterVolumeSpecName: "cacerts") pod "73aeb98c-a0c3-4989-9c5f-1c5a448d345a" (UID: "73aeb98c-a0c3-4989-9c5f-1c5a448d345a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:05:29.542219 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.542190    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-local-certs" (OuterVolumeSpecName: "local-certs") pod "73aeb98c-a0c3-4989-9c5f-1c5a448d345a" (UID: "73aeb98c-a0c3-4989-9c5f-1c5a448d345a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:05:29.635778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.635613    2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-kubeconfig\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:05:29.635778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.635650    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-ca-configmap\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:05:29.635778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.635666    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-token\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:05:29.635778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.635681    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctbwf\" (UniqueName: \"kubernetes.io/projected/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-kube-api-access-ctbwf\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:05:29.635778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.635695    2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-cacerts\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:05:29.635778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.635708    2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-local-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:05:29.635778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:29.635722    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73aeb98c-a0c3-4989-9c5f-1c5a448d345a-istio-csr-dns-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:05:30.443094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:30.443057    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"
Jul 28 18:05:30.443540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:30.443093    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm" event={"ID":"73aeb98c-a0c3-4989-9c5f-1c5a448d345a","Type":"ContainerDied","Data":"79bd218c989d638b4a37b2caa125c5bf0aa4e8cc452e9a003999f14b27c0b159"}
Jul 28 18:05:30.443540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:30.443167    2575 scope.go:117] "RemoveContainer" containerID="09c62311c0e183bb538e79e62bff72ec7ed225c27d20b39c0706b3ae28712c60"
Jul 28 18:05:30.507050 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:30.507009    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"]
Jul 28 18:05:30.519714 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:30.519666    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-ddkfm"]
Jul 28 18:05:31.389225 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:31.389135    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-t5252"
Jul 28 18:05:31.391453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:31.391425    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7j5jt"
Jul 28 18:05:32.400298 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:32.400269    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73aeb98c-a0c3-4989-9c5f-1c5a448d345a" path="/var/lib/kubelet/pods/73aeb98c-a0c3-4989-9c5f-1c5a448d345a/volumes"
Jul 28 18:05:32.457216 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:32.457183    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d" event={"ID":"fd0ad590-6443-4893-8bf9-6f7be58ecad8","Type":"ContainerStarted","Data":"60e169f4c36fc188f22bfb1e228d00f4fca973e6a912d91336a6035fa2372c1c"}
Jul 28 18:05:32.457432 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:32.457298    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:05:32.458843 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:32.458819    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj" event={"ID":"fb7e6d94-5676-4c6f-b149-6173d4d563a1","Type":"ContainerStarted","Data":"a87ad1028ddfbaade0f61cbc9590eb63e6840f0c378c6a6f635e27e2e35637cf"}
Jul 28 18:05:32.458981 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:32.458911    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:05:32.493794 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:32.493736    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d" podStartSLOduration=2.157210311 podStartE2EDuration="5.493719562s" totalImagesPullingTime="3.336509251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:32.493014542 +0000 UTC m=+506.638567517" watchObservedRunningTime="2026-07-28 18:05:32.493719562 +0000 UTC m=+506.639272536"
Jul 28 18:05:32.541868 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:05:32.541797    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj" podStartSLOduration=2.3254513709999998 podStartE2EDuration="5.54177971s" totalImagesPullingTime="3.216328339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:32.541470337 +0000 UTC m=+506.687023310" watchObservedRunningTime="2026-07-28 18:05:32.54177971 +0000 UTC m=+506.687332685"
Jul 28 18:06:03.465475 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:03.465392    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b5jmj"
Jul 28 18:06:03.468448 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:03.468427    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7qx6d"
Jul 28 18:06:14.524107 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.524067    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ch2fd"]
Jul 28 18:06:14.524566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.524370    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aeb98c-a0c3-4989-9c5f-1c5a448d345a" containerName="discovery"
Jul 28 18:06:14.524566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.524381    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aeb98c-a0c3-4989-9c5f-1c5a448d345a" containerName="discovery"
Jul 28 18:06:14.524566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.524442    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="73aeb98c-a0c3-4989-9c5f-1c5a448d345a" containerName="discovery"
Jul 28 18:06:14.535052 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.535021    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd"
Jul 28 18:06:14.538206 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.538181    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 18:06:14.539211 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.539170    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-whsg2\""
Jul 28 18:06:14.539211 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.539171    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 18:06:14.549758 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.549723    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ch2fd"]
Jul 28 18:06:14.621839 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.621799    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p67bm\" (UniqueName: \"kubernetes.io/projected/343a542d-c945-46e9-a2c3-00400c83a500-kube-api-access-p67bm\") pod \"kuadrant-operator-catalog-ch2fd\" (UID: \"343a542d-c945-46e9-a2c3-00400c83a500\") " pod="kuadrant-system/kuadrant-operator-catalog-ch2fd"
Jul 28 18:06:14.722361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.722328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p67bm\" (UniqueName: \"kubernetes.io/projected/343a542d-c945-46e9-a2c3-00400c83a500-kube-api-access-p67bm\") pod \"kuadrant-operator-catalog-ch2fd\" (UID: \"343a542d-c945-46e9-a2c3-00400c83a500\") " pod="kuadrant-system/kuadrant-operator-catalog-ch2fd"
Jul 28 18:06:14.731826 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.731793    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p67bm\" (UniqueName: \"kubernetes.io/projected/343a542d-c945-46e9-a2c3-00400c83a500-kube-api-access-p67bm\") pod \"kuadrant-operator-catalog-ch2fd\" (UID: \"343a542d-c945-46e9-a2c3-00400c83a500\") " pod="kuadrant-system/kuadrant-operator-catalog-ch2fd"
Jul 28 18:06:14.846148 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.846056    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd"
Jul 28 18:06:14.877091 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.877053    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ch2fd"]
Jul 28 18:06:14.994233 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:14.994078    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ch2fd"]
Jul 28 18:06:14.997008 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:06:14.996972    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343a542d_c945_46e9_a2c3_00400c83a500.slice/crio-efb46a3c554ed7b5495c1e65bbd157d0825ff866a34c2e129c739e3bed76cf2b WatchSource:0}: Error finding container efb46a3c554ed7b5495c1e65bbd157d0825ff866a34c2e129c739e3bed76cf2b: Status 404 returned error can't find the container with id efb46a3c554ed7b5495c1e65bbd157d0825ff866a34c2e129c739e3bed76cf2b
Jul 28 18:06:15.097094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.097006    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qckb"]
Jul 28 18:06:15.099908 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.099890    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:15.120692 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.120658    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qckb"]
Jul 28 18:06:15.125542 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.125509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nfs9\" (UniqueName: \"kubernetes.io/projected/207536a4-cbcc-4960-97c7-c9316d3a7863-kube-api-access-8nfs9\") pod \"kuadrant-operator-catalog-5qckb\" (UID: \"207536a4-cbcc-4960-97c7-c9316d3a7863\") " pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:15.226443 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.226405    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfs9\" (UniqueName: \"kubernetes.io/projected/207536a4-cbcc-4960-97c7-c9316d3a7863-kube-api-access-8nfs9\") pod \"kuadrant-operator-catalog-5qckb\" (UID: \"207536a4-cbcc-4960-97c7-c9316d3a7863\") " pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:15.239381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.239345    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfs9\" (UniqueName: \"kubernetes.io/projected/207536a4-cbcc-4960-97c7-c9316d3a7863-kube-api-access-8nfs9\") pod \"kuadrant-operator-catalog-5qckb\" (UID: \"207536a4-cbcc-4960-97c7-c9316d3a7863\") " pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:15.409586 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.409487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:15.552526 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.552490    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5qckb"]
Jul 28 18:06:15.555293 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:06:15.555265    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207536a4_cbcc_4960_97c7_c9316d3a7863.slice/crio-cd6017f7bc153a664c3c11f71e53cbda8c3e9d2b56664cb8611efd528f202972 WatchSource:0}: Error finding container cd6017f7bc153a664c3c11f71e53cbda8c3e9d2b56664cb8611efd528f202972: Status 404 returned error can't find the container with id cd6017f7bc153a664c3c11f71e53cbda8c3e9d2b56664cb8611efd528f202972
Jul 28 18:06:15.644563 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.644529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5qckb" event={"ID":"207536a4-cbcc-4960-97c7-c9316d3a7863","Type":"ContainerStarted","Data":"cd6017f7bc153a664c3c11f71e53cbda8c3e9d2b56664cb8611efd528f202972"}
Jul 28 18:06:15.646203 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:15.646169    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd" event={"ID":"343a542d-c945-46e9-a2c3-00400c83a500","Type":"ContainerStarted","Data":"efb46a3c554ed7b5495c1e65bbd157d0825ff866a34c2e129c739e3bed76cf2b"}
Jul 28 18:06:17.657544 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.657507    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5qckb" event={"ID":"207536a4-cbcc-4960-97c7-c9316d3a7863","Type":"ContainerStarted","Data":"849050f04a69908d1dd0f151eab594fd9a51726cbe6ed4e6d150ac3621773fb1"}
Jul 28 18:06:17.659065 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.659031    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd" event={"ID":"343a542d-c945-46e9-a2c3-00400c83a500","Type":"ContainerStarted","Data":"d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df"}
Jul 28 18:06:17.659193 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.659082    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd" podUID="343a542d-c945-46e9-a2c3-00400c83a500" containerName="registry-server" containerID="cri-o://d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df" gracePeriod=2
Jul 28 18:06:17.691357 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.691296    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5qckb" podStartSLOduration=1.155615886 podStartE2EDuration="2.691277418s" totalImagesPullingTime="1.535661532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:17.69088479 +0000 UTC m=+551.836437763" watchObservedRunningTime="2026-07-28 18:06:17.691277418 +0000 UTC m=+551.836830392"
Jul 28 18:06:17.716069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.716016    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd" podStartSLOduration=1.6263647639999999 podStartE2EDuration="3.715998095s" totalImagesPullingTime="2.089633331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:17.715999631 +0000 UTC m=+551.861552606" watchObservedRunningTime="2026-07-28 18:06:17.715998095 +0000 UTC m=+551.861551112"
Jul 28 18:06:17.906695 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.906667    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd"
Jul 28 18:06:17.946055 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.945968    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p67bm\" (UniqueName: \"kubernetes.io/projected/343a542d-c945-46e9-a2c3-00400c83a500-kube-api-access-p67bm\") pod \"343a542d-c945-46e9-a2c3-00400c83a500\" (UID: \"343a542d-c945-46e9-a2c3-00400c83a500\") "
Jul 28 18:06:17.948166 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:17.948132    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/343a542d-c945-46e9-a2c3-00400c83a500-kube-api-access-p67bm" (OuterVolumeSpecName: "kube-api-access-p67bm") pod "343a542d-c945-46e9-a2c3-00400c83a500" (UID: "343a542d-c945-46e9-a2c3-00400c83a500"). InnerVolumeSpecName "kube-api-access-p67bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:06:18.046624 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.046582    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p67bm\" (UniqueName: \"kubernetes.io/projected/343a542d-c945-46e9-a2c3-00400c83a500-kube-api-access-p67bm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:06:18.664976 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.664881    2575 generic.go:356] "Generic (PLEG): container finished" podID="343a542d-c945-46e9-a2c3-00400c83a500" containerID="d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df" exitCode=0
Jul 28 18:06:18.664976 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.664922    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd"
Jul 28 18:06:18.664976 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.664966    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd" event={"ID":"343a542d-c945-46e9-a2c3-00400c83a500","Type":"ContainerDied","Data":"d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df"}
Jul 28 18:06:18.665489 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.665017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ch2fd" event={"ID":"343a542d-c945-46e9-a2c3-00400c83a500","Type":"ContainerDied","Data":"efb46a3c554ed7b5495c1e65bbd157d0825ff866a34c2e129c739e3bed76cf2b"}
Jul 28 18:06:18.665489 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.665047    2575 scope.go:117] "RemoveContainer" containerID="d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df"
Jul 28 18:06:18.674514 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.674489    2575 scope.go:117] "RemoveContainer" containerID="d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df"
Jul 28 18:06:18.674843 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:06:18.674821    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df\": container with ID starting with d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df not found: ID does not exist" containerID="d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df"
Jul 28 18:06:18.674947 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.674864    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df"} err="failed to get container status \"d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df\": rpc error: code = NotFound desc = could not find container \"d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df\": container with ID starting with d28b307fbe9a8e4f0caf56208729c826cfe4c702753ccbce56348798a12910df not found: ID does not exist"
Jul 28 18:06:18.683924 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.683847    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ch2fd"]
Jul 28 18:06:18.685429 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:18.685401    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ch2fd"]
Jul 28 18:06:20.399532 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:20.399491    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="343a542d-c945-46e9-a2c3-00400c83a500" path="/var/lib/kubelet/pods/343a542d-c945-46e9-a2c3-00400c83a500/volumes"
Jul 28 18:06:25.410040 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:25.410004    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:25.410458 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:25.410064    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:25.432418 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:25.432389    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:25.717703 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:25.717624    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5qckb"
Jul 28 18:06:46.515795 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.515759    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"]
Jul 28 18:06:46.516207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.516080    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="343a542d-c945-46e9-a2c3-00400c83a500" containerName="registry-server"
Jul 28 18:06:46.516207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.516091    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="343a542d-c945-46e9-a2c3-00400c83a500" containerName="registry-server"
Jul 28 18:06:46.516207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.516149    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="343a542d-c945-46e9-a2c3-00400c83a500" containerName="registry-server"
Jul 28 18:06:46.518278 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.518261    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:06:46.524306 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.524279    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ks626\""
Jul 28 18:06:46.535081 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.535047    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"]
Jul 28 18:06:46.589329 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.589287    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtvqb\" (UniqueName: \"kubernetes.io/projected/6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa-kube-api-access-qtvqb\") pod \"limitador-operator-controller-manager-859d7b78bd-x86vq\" (UID: \"6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:06:46.690499 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.690444    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtvqb\" (UniqueName: \"kubernetes.io/projected/6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa-kube-api-access-qtvqb\") pod \"limitador-operator-controller-manager-859d7b78bd-x86vq\" (UID: \"6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:06:46.706597 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.706567    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtvqb\" (UniqueName: \"kubernetes.io/projected/6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa-kube-api-access-qtvqb\") pod \"limitador-operator-controller-manager-859d7b78bd-x86vq\" (UID: \"6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:06:46.828935 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:46.828801    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:06:47.014588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:47.014549    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"]
Jul 28 18:06:47.020214 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:06:47.020182    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0ff89d_d2e8_483c_83ca_92bc2c5f8bfa.slice/crio-e64486b1a4ab9ae8c67e6c06af994fa75ac76f98e9a9c892320af4253d9713bc WatchSource:0}: Error finding container e64486b1a4ab9ae8c67e6c06af994fa75ac76f98e9a9c892320af4253d9713bc: Status 404 returned error can't find the container with id e64486b1a4ab9ae8c67e6c06af994fa75ac76f98e9a9c892320af4253d9713bc
Jul 28 18:06:47.796762 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:47.796709    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq" event={"ID":"6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa","Type":"ContainerStarted","Data":"e64486b1a4ab9ae8c67e6c06af994fa75ac76f98e9a9c892320af4253d9713bc"}
Jul 28 18:06:49.809824 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:49.809725    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq" event={"ID":"6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa","Type":"ContainerStarted","Data":"91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b"}
Jul 28 18:06:49.810285 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:49.809823    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:06:49.840879 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:49.840803    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq" podStartSLOduration=1.729560489 podStartE2EDuration="3.840784485s" totalImagesPullingTime="2.111223996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:49.837952176 +0000 UTC m=+583.983505147" watchObservedRunningTime="2026-07-28 18:06:49.840784485 +0000 UTC m=+583.986337459"
Jul 28 18:06:51.710438 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.710401    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"]
Jul 28 18:06:51.712803 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.712785    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:51.717669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.717646    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8zvj5\""
Jul 28 18:06:51.730743 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.730709    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"]
Jul 28 18:06:51.840945 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.840901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcsxm\" (UniqueName: \"kubernetes.io/projected/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-kube-api-access-tcsxm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:51.841138 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.840960    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:51.941720 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.941680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcsxm\" (UniqueName: \"kubernetes.io/projected/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-kube-api-access-tcsxm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:51.941931 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.941737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:51.942243 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.942224    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:51.958562 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:51.958513    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcsxm\" (UniqueName: \"kubernetes.io/projected/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-kube-api-access-tcsxm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:52.022735 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:52.022698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:52.180960 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:52.180933    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"]
Jul 28 18:06:52.184048 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:06:52.184021    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b21179_ffe8_4c6d_a9ee_7f1f0aee9960.slice/crio-d0b6e9ff5de42b51ce3d6b7aa01150537db6048863fc231b81228b33e02d215f WatchSource:0}: Error finding container d0b6e9ff5de42b51ce3d6b7aa01150537db6048863fc231b81228b33e02d215f: Status 404 returned error can't find the container with id d0b6e9ff5de42b51ce3d6b7aa01150537db6048863fc231b81228b33e02d215f
Jul 28 18:06:52.829408 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:52.829365    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj" event={"ID":"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960","Type":"ContainerStarted","Data":"d0b6e9ff5de42b51ce3d6b7aa01150537db6048863fc231b81228b33e02d215f"}
Jul 28 18:06:54.653993 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.653956    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"]
Jul 28 18:06:54.656337 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.656319    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"
Jul 28 18:06:54.661091 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.661056    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k98ml\""
Jul 28 18:06:54.661307 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.661292    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 18:06:54.682653 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.682622    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"]
Jul 28 18:06:54.768423 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.768386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx6fs\" (UniqueName: \"kubernetes.io/projected/fcdfba35-4f17-44e5-a487-bc50d98dd536-kube-api-access-rx6fs\") pod \"dns-operator-controller-manager-864dd78c4c-m4cbd\" (UID: \"fcdfba35-4f17-44e5-a487-bc50d98dd536\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"
Jul 28 18:06:54.869308 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.869203    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx6fs\" (UniqueName: \"kubernetes.io/projected/fcdfba35-4f17-44e5-a487-bc50d98dd536-kube-api-access-rx6fs\") pod \"dns-operator-controller-manager-864dd78c4c-m4cbd\" (UID: \"fcdfba35-4f17-44e5-a487-bc50d98dd536\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"
Jul 28 18:06:54.894150 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.894117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx6fs\" (UniqueName: \"kubernetes.io/projected/fcdfba35-4f17-44e5-a487-bc50d98dd536-kube-api-access-rx6fs\") pod \"dns-operator-controller-manager-864dd78c4c-m4cbd\" (UID: \"fcdfba35-4f17-44e5-a487-bc50d98dd536\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"
Jul 28 18:06:54.967824 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:54.967724    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"
Jul 28 18:06:55.118396 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:55.118294    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"]
Jul 28 18:06:55.121405 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:06:55.121368    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdfba35_4f17_44e5_a487_bc50d98dd536.slice/crio-c7b69f30ed588604e0e3e124073ad2a5213a2f56635e3233ed7beace4f9e4600 WatchSource:0}: Error finding container c7b69f30ed588604e0e3e124073ad2a5213a2f56635e3233ed7beace4f9e4600: Status 404 returned error can't find the container with id c7b69f30ed588604e0e3e124073ad2a5213a2f56635e3233ed7beace4f9e4600
Jul 28 18:06:55.846414 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:55.846334    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd" event={"ID":"fcdfba35-4f17-44e5-a487-bc50d98dd536","Type":"ContainerStarted","Data":"c7b69f30ed588604e0e3e124073ad2a5213a2f56635e3233ed7beace4f9e4600"}
Jul 28 18:06:58.863469 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:58.863432    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj" event={"ID":"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960","Type":"ContainerStarted","Data":"c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4"}
Jul 28 18:06:58.863977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:58.863536    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:06:58.865040 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:58.865013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd" event={"ID":"fcdfba35-4f17-44e5-a487-bc50d98dd536","Type":"ContainerStarted","Data":"3c3fd760db783f74242c2b5516dcb9972721c03ed8db593fa2f886dad99b81d2"}
Jul 28 18:06:58.865139 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:58.865123    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"
Jul 28 18:06:58.886933 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:58.886874    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj" podStartSLOduration=1.8721407540000001 podStartE2EDuration="7.886837435s" totalImagesPullingTime="6.014696681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:58.885720566 +0000 UTC m=+593.031273530" watchObservedRunningTime="2026-07-28 18:06:58.886837435 +0000 UTC m=+593.032390410"
Jul 28 18:06:58.910528 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:06:58.910474    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd" podStartSLOduration=1.843719903 podStartE2EDuration="4.910455142s" totalImagesPullingTime="3.066735239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:58.909040512 +0000 UTC m=+593.054593487" watchObservedRunningTime="2026-07-28 18:06:58.910455142 +0000 UTC m=+593.056008116"
Jul 28 18:07:00.816215 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:00.816186    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:01.398818 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.398784    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"]
Jul 28 18:07:01.401116 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.401098    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.409110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.409073    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 18:07:01.409110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.409100    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zprdz\""
Jul 28 18:07:01.423844 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.423808    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"]
Jul 28 18:07:01.424387 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.424370    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 18:07:01.531578 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.531533    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f7152f1-425c-4ec5-b714-c242113bbb14-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.531578 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.531574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rvj4\" (UniqueName: \"kubernetes.io/projected/8f7152f1-425c-4ec5-b714-c242113bbb14-kube-api-access-5rvj4\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.531836 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.531742    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f7152f1-425c-4ec5-b714-c242113bbb14-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.633139 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.633098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f7152f1-425c-4ec5-b714-c242113bbb14-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.633139 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.633141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvj4\" (UniqueName: \"kubernetes.io/projected/8f7152f1-425c-4ec5-b714-c242113bbb14-kube-api-access-5rvj4\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.633376 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.633179    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f7152f1-425c-4ec5-b714-c242113bbb14-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.633807 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.633782    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f7152f1-425c-4ec5-b714-c242113bbb14-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.635755 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.635725    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f7152f1-425c-4ec5-b714-c242113bbb14-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.648010 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.647984    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvj4\" (UniqueName: \"kubernetes.io/projected/8f7152f1-425c-4ec5-b714-c242113bbb14-kube-api-access-5rvj4\") pod \"kuadrant-console-plugin-66b5774985-jn96v\" (UID: \"8f7152f1-425c-4ec5-b714-c242113bbb14\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.711212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.711115    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"
Jul 28 18:07:01.858914 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.858891    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v"]
Jul 28 18:07:01.861834 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:07:01.861809    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f7152f1_425c_4ec5_b714_c242113bbb14.slice/crio-36abd519587f0072b48c5c34cc297aed8c3288114df5354d7e35eed870e8c7cd WatchSource:0}: Error finding container 36abd519587f0072b48c5c34cc297aed8c3288114df5354d7e35eed870e8c7cd: Status 404 returned error can't find the container with id 36abd519587f0072b48c5c34cc297aed8c3288114df5354d7e35eed870e8c7cd
Jul 28 18:07:01.885969 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:01.885936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v" event={"ID":"8f7152f1-425c-4ec5-b714-c242113bbb14","Type":"ContainerStarted","Data":"36abd519587f0072b48c5c34cc297aed8c3288114df5354d7e35eed870e8c7cd"}
Jul 28 18:07:07.181786 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:07.181749    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:07:07.182831 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:07.182804    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:07:09.872452 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:09.872421    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:09.872902 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:09.872515    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m4cbd"
Jul 28 18:07:11.804159 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.804125    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"]
Jul 28 18:07:11.819517 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.819484    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.824565 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.824533    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xr2p\" (UniqueName: \"kubernetes.io/projected/f0e6e030-fd3a-4261-9d3a-7a805f550684-kube-api-access-7xr2p\") pod \"kuadrant-operator-controller-manager-86878c769f-f96kh\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.824730 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.824580    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e6e030-fd3a-4261-9d3a-7a805f550684-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f96kh\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.838932 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.838897    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"]
Jul 28 18:07:11.925549 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.925513    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xr2p\" (UniqueName: \"kubernetes.io/projected/f0e6e030-fd3a-4261-9d3a-7a805f550684-kube-api-access-7xr2p\") pod \"kuadrant-operator-controller-manager-86878c769f-f96kh\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.925549 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.925555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e6e030-fd3a-4261-9d3a-7a805f550684-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f96kh\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.926089 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.926067    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e6e030-fd3a-4261-9d3a-7a805f550684-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f96kh\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.941180 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.941146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xr2p\" (UniqueName: \"kubernetes.io/projected/f0e6e030-fd3a-4261-9d3a-7a805f550684-kube-api-access-7xr2p\") pod \"kuadrant-operator-controller-manager-86878c769f-f96kh\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.945525 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.945491    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"]
Jul 28 18:07:11.945768 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.945696    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" containerName="manager" containerID="cri-o://c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4" gracePeriod=2
Jul 28 18:07:11.965226 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.965188    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"]
Jul 28 18:07:11.973516 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.973482    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"]
Jul 28 18:07:11.973828 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.973811    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:11.986606 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.986530    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"]
Jul 28 18:07:11.992491 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.992456    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"]
Jul 28 18:07:11.993016 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.992990    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" containerName="manager"
Jul 28 18:07:11.993016 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.993015    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" containerName="manager"
Jul 28 18:07:11.993212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.993145    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" containerName="manager"
Jul 28 18:07:11.995600 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:11.995576    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.002594 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:12.002550    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:12.015983 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.015955    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"]
Jul 28 18:07:12.016426 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.016395    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" containerName="manager" containerID="cri-o://91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b" gracePeriod=2
Jul 28 18:07:12.019614 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.019581    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"]
Jul 28 18:07:12.026341 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.026295    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fzjk\" (UniqueName: \"kubernetes.io/projected/e513629d-958d-46f8-8eb1-125c76420824-kube-api-access-5fzjk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7qg8l\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.026495 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.026384    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e513629d-958d-46f8-8eb1-125c76420824-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7qg8l\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.028490 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.028363    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"]
Jul 28 18:07:12.035164 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.035103    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"]
Jul 28 18:07:12.035509 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.035493    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" containerName="manager"
Jul 28 18:07:12.035509 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.035510    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" containerName="manager"
Jul 28 18:07:12.035654 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.035575    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" containerName="manager"
Jul 28 18:07:12.037969 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.037943    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:12.055908 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:12.055784    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x86vq\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:12.058233 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:12.058195    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:12.065939 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.065907    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"]
Jul 28 18:07:12.127422 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.127384    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fzjk\" (UniqueName: \"kubernetes.io/projected/e513629d-958d-46f8-8eb1-125c76420824-kube-api-access-5fzjk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7qg8l\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.127610 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.127445    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd950a52-7d1d-40d6-b78d-76393c222427-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fpjqm\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:12.127610 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.127492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e513629d-958d-46f8-8eb1-125c76420824-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7qg8l\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.127610 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.127517    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pf6l\" (UniqueName: \"kubernetes.io/projected/dd950a52-7d1d-40d6-b78d-76393c222427-kube-api-access-2pf6l\") pod \"kuadrant-operator-controller-manager-86878c769f-fpjqm\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:12.127966 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.127938    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e513629d-958d-46f8-8eb1-125c76420824-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7qg8l\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.160695 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.160656    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fzjk\" (UniqueName: \"kubernetes.io/projected/e513629d-958d-46f8-8eb1-125c76420824-kube-api-access-5fzjk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7qg8l\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.228415 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.228373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd950a52-7d1d-40d6-b78d-76393c222427-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fpjqm\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:12.228601 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.228436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pf6l\" (UniqueName: \"kubernetes.io/projected/dd950a52-7d1d-40d6-b78d-76393c222427-kube-api-access-2pf6l\") pod \"kuadrant-operator-controller-manager-86878c769f-fpjqm\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:12.228806 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.228784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd950a52-7d1d-40d6-b78d-76393c222427-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fpjqm\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:12.239055 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.239018    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pf6l\" (UniqueName: \"kubernetes.io/projected/dd950a52-7d1d-40d6-b78d-76393c222427-kube-api-access-2pf6l\") pod \"kuadrant-operator-controller-manager-86878c769f-fpjqm\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:12.324947 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.324832    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:12.390226 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:12.390180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:24.572511 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.572461    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"]
Jul 28 18:07:24.573909 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:07:24.573883    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd950a52_7d1d_40d6_b78d_76393c222427.slice/crio-567c213882d0ec72ab477155c702fbaca606cada721c21a5bd2536d74ce8fe1f WatchSource:0}: Error finding container 567c213882d0ec72ab477155c702fbaca606cada721c21a5bd2536d74ce8fe1f: Status 404 returned error can't find the container with id 567c213882d0ec72ab477155c702fbaca606cada721c21a5bd2536d74ce8fe1f
Jul 28 18:07:24.594803 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.594777    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"]
Jul 28 18:07:24.597469 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.597433    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:24.598214 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:07:24.598189    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode513629d_958d_46f8_8eb1_125c76420824.slice/crio-b3afb9c5941c07438684ce3f37b1ce906b249e865d6e75fe58b2e2770584c9d4 WatchSource:0}: Error finding container b3afb9c5941c07438684ce3f37b1ce906b249e865d6e75fe58b2e2770584c9d4: Status 404 returned error can't find the container with id b3afb9c5941c07438684ce3f37b1ce906b249e865d6e75fe58b2e2770584c9d4
Jul 28 18:07:24.599665 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:24.599628    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:24.602588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.602525    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:24.605261 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:24.605225    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x86vq\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:24.607347 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:24.607280    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:24.640587 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.640563    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtvqb\" (UniqueName: \"kubernetes.io/projected/6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa-kube-api-access-qtvqb\") pod \"6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa\" (UID: \"6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa\") "
Jul 28 18:07:24.640719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.640594    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcsxm\" (UniqueName: \"kubernetes.io/projected/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-kube-api-access-tcsxm\") pod \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") "
Jul 28 18:07:24.640719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.640638    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-extensions-socket-volume\") pod \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\" (UID: \"07b21179-ffe8-4c6d-a9ee-7f1f0aee9960\") "
Jul 28 18:07:24.641300 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.641268    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" (UID: "07b21179-ffe8-4c6d-a9ee-7f1f0aee9960"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:07:24.642830 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.642799    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-kube-api-access-tcsxm" (OuterVolumeSpecName: "kube-api-access-tcsxm") pod "07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" (UID: "07b21179-ffe8-4c6d-a9ee-7f1f0aee9960"). InnerVolumeSpecName "kube-api-access-tcsxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:07:24.643189 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.643126    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa-kube-api-access-qtvqb" (OuterVolumeSpecName: "kube-api-access-qtvqb") pod "6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" (UID: "6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa"). InnerVolumeSpecName "kube-api-access-qtvqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:07:24.741819 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.741775    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtvqb\" (UniqueName: \"kubernetes.io/projected/6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa-kube-api-access-qtvqb\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:07:24.741819 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.741813    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcsxm\" (UniqueName: \"kubernetes.io/projected/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-kube-api-access-tcsxm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:07:24.741819 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:24.741823    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960-extensions-socket-volume\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:07:25.009247 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.009209    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v" event={"ID":"8f7152f1-425c-4ec5-b714-c242113bbb14","Type":"ContainerStarted","Data":"80a6320aa75868db48133af0c1a540acc22177aca8336885dcf9900f2fbb7035"}
Jul 28 18:07:25.010692 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.010655    2575 generic.go:356] "Generic (PLEG): container finished" podID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" containerID="91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b" exitCode=0
Jul 28 18:07:25.010868 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.010698    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:25.010868 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.010701    2575 scope.go:117] "RemoveContainer" containerID="91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b"
Jul 28 18:07:25.012689 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.012661    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l" event={"ID":"e513629d-958d-46f8-8eb1-125c76420824","Type":"ContainerStarted","Data":"be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d"}
Jul 28 18:07:25.012807 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.012726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l" event={"ID":"e513629d-958d-46f8-8eb1-125c76420824","Type":"ContainerStarted","Data":"b3afb9c5941c07438684ce3f37b1ce906b249e865d6e75fe58b2e2770584c9d4"}
Jul 28 18:07:25.012807 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.012791    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:25.014802 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.014777    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm" event={"ID":"dd950a52-7d1d-40d6-b78d-76393c222427","Type":"ContainerStarted","Data":"303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d"}
Jul 28 18:07:25.014961 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.014831    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm" event={"ID":"dd950a52-7d1d-40d6-b78d-76393c222427","Type":"ContainerStarted","Data":"567c213882d0ec72ab477155c702fbaca606cada721c21a5bd2536d74ce8fe1f"}
Jul 28 18:07:25.014961 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.014906    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:25.016803 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.016776    2575 generic.go:356] "Generic (PLEG): container finished" podID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" containerID="c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4" exitCode=0
Jul 28 18:07:25.016933 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.016813    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:25.020809 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.020789    2575 scope.go:117] "RemoveContainer" containerID="91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b"
Jul 28 18:07:25.021124 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:25.021100    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b\": container with ID starting with 91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b not found: ID does not exist" containerID="91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b"
Jul 28 18:07:25.021197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.021133    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b"} err="failed to get container status \"91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b\": rpc error: code = NotFound desc = could not find container \"91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b\": container with ID starting with 91a9bd9b390695c3e7e268962e548b50004021845ac2ca485daaf52439b7a62b not found: ID does not exist"
Jul 28 18:07:25.021197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.021151    2575 scope.go:117] "RemoveContainer" containerID="c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4"
Jul 28 18:07:25.027824 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.027777    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jn96v" podStartSLOduration=1.269151209 podStartE2EDuration="24.027758334s" totalImagesPullingTime="22.758607125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:25.026394236 +0000 UTC m=+619.171947210" watchObservedRunningTime="2026-07-28 18:07:25.027758334 +0000 UTC m=+619.173311308"
Jul 28 18:07:25.029922 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:25.029815    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x86vq\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:25.032050 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:25.031981    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:25.034581 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.034333    2575 scope.go:117] "RemoveContainer" containerID="c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4"
Jul 28 18:07:25.034581 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:25.034537    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x86vq\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:25.035507 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:25.035439    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4\": container with ID starting with c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4 not found: ID does not exist" containerID="c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4"
Jul 28 18:07:25.035507 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.035477    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4"} err="failed to get container status \"c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4\": rpc error: code = NotFound desc = could not find container \"c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4\": container with ID starting with c2a2bbe0b0ddccfcdbbe1e00670d4329bdc54add533dabf13f0d5bca789ffcf4 not found: ID does not exist"
Jul 28 18:07:25.036553 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:25.036515    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:25.064110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.064035    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm" podStartSLOduration=14.064013098 podStartE2EDuration="14.064013098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:25.062431678 +0000 UTC m=+619.207984651" watchObservedRunningTime="2026-07-28 18:07:25.064013098 +0000 UTC m=+619.209566075"
Jul 28 18:07:25.085745 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:25.085688    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l" podStartSLOduration=14.085672246 podStartE2EDuration="14.085672246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:25.084052497 +0000 UTC m=+619.229605484" watchObservedRunningTime="2026-07-28 18:07:25.085672246 +0000 UTC m=+619.231225220"
Jul 28 18:07:26.398015 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:26.397979    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x86vq\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x86vq"
Jul 28 18:07:26.399522 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:26.399499    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" path="/var/lib/kubelet/pods/07b21179-ffe8-4c6d-a9ee-7f1f0aee9960/volumes"
Jul 28 18:07:26.399825 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:26.399813    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa" path="/var/lib/kubelet/pods/6a0ff89d-d2e8-483c-83ca-92bc2c5f8bfa/volumes"
Jul 28 18:07:26.400937 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:26.400915    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zzncj\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="07b21179-ffe8-4c6d-a9ee-7f1f0aee9960" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zzncj"
Jul 28 18:07:26.935730 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:26.935685    2575 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-f96kh_kuadrant-system_f0e6e030-fd3a-4261-9d3a-7a805f550684_0(5aeacada2c0d431dcaa0e72458424b9e6dc772857416a7b8169e3a648f6b687a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f96kh to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 28 18:07:26.935809 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:26.935786    2575 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-f96kh_kuadrant-system_f0e6e030-fd3a-4261-9d3a-7a805f550684_0(5aeacada2c0d431dcaa0e72458424b9e6dc772857416a7b8169e3a648f6b687a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f96kh to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:27.027737 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.027703    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:27.030289 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:27.030258    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-f96kh\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="f0e6e030-fd3a-4261-9d3a-7a805f550684" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:27.172916 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.172802    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:27.175551 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:27.175521    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-f96kh\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="f0e6e030-fd3a-4261-9d3a-7a805f550684" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:27.263390 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.263352    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e6e030-fd3a-4261-9d3a-7a805f550684-extensions-socket-volume\") pod \"f0e6e030-fd3a-4261-9d3a-7a805f550684\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") "
Jul 28 18:07:27.263554 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.263470    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xr2p\" (UniqueName: \"kubernetes.io/projected/f0e6e030-fd3a-4261-9d3a-7a805f550684-kube-api-access-7xr2p\") pod \"f0e6e030-fd3a-4261-9d3a-7a805f550684\" (UID: \"f0e6e030-fd3a-4261-9d3a-7a805f550684\") "
Jul 28 18:07:27.263653 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.263629    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e6e030-fd3a-4261-9d3a-7a805f550684-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f0e6e030-fd3a-4261-9d3a-7a805f550684" (UID: "f0e6e030-fd3a-4261-9d3a-7a805f550684"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:07:27.265630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.265599    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e6e030-fd3a-4261-9d3a-7a805f550684-kube-api-access-7xr2p" (OuterVolumeSpecName: "kube-api-access-7xr2p") pod "f0e6e030-fd3a-4261-9d3a-7a805f550684" (UID: "f0e6e030-fd3a-4261-9d3a-7a805f550684"). InnerVolumeSpecName "kube-api-access-7xr2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:07:27.364794 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.364701    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xr2p\" (UniqueName: \"kubernetes.io/projected/f0e6e030-fd3a-4261-9d3a-7a805f550684-kube-api-access-7xr2p\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:07:27.364794 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:27.364734    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0e6e030-fd3a-4261-9d3a-7a805f550684-extensions-socket-volume\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:07:28.032165 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:28.032133    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:28.035028 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:28.034994    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-f96kh\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="f0e6e030-fd3a-4261-9d3a-7a805f550684" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:28.040292 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:28.040261    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-f96kh\" is forbidden: User \"system:node:ip-10-0-132-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-23.ec2.internal' and this object" podUID="f0e6e030-fd3a-4261-9d3a-7a805f550684" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f96kh"
Jul 28 18:07:28.399939 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:28.399832    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e6e030-fd3a-4261-9d3a-7a805f550684" path="/var/lib/kubelet/pods/f0e6e030-fd3a-4261-9d3a-7a805f550684/volumes"
Jul 28 18:07:36.026863 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:36.026814    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:07:36.027265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:36.027051    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:36.117144 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:36.117105    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"]
Jul 28 18:07:36.117454 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:36.117425    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l" podUID="e513629d-958d-46f8-8eb1-125c76420824" containerName="manager" containerID="cri-o://be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d" gracePeriod=10
Jul 28 18:07:36.913539 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:36.913507    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:37.058069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.057974    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e513629d-958d-46f8-8eb1-125c76420824-extensions-socket-volume\") pod \"e513629d-958d-46f8-8eb1-125c76420824\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") "
Jul 28 18:07:37.058069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.058070    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fzjk\" (UniqueName: \"kubernetes.io/projected/e513629d-958d-46f8-8eb1-125c76420824-kube-api-access-5fzjk\") pod \"e513629d-958d-46f8-8eb1-125c76420824\" (UID: \"e513629d-958d-46f8-8eb1-125c76420824\") "
Jul 28 18:07:37.058565 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.058443    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e513629d-958d-46f8-8eb1-125c76420824-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e513629d-958d-46f8-8eb1-125c76420824" (UID: "e513629d-958d-46f8-8eb1-125c76420824"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:07:37.060271 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.060245    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e513629d-958d-46f8-8eb1-125c76420824-kube-api-access-5fzjk" (OuterVolumeSpecName: "kube-api-access-5fzjk") pod "e513629d-958d-46f8-8eb1-125c76420824" (UID: "e513629d-958d-46f8-8eb1-125c76420824"). InnerVolumeSpecName "kube-api-access-5fzjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:07:37.085745 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.085710    2575 generic.go:356] "Generic (PLEG): container finished" podID="e513629d-958d-46f8-8eb1-125c76420824" containerID="be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d" exitCode=0
Jul 28 18:07:37.085941 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.085763    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"
Jul 28 18:07:37.085941 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.085805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l" event={"ID":"e513629d-958d-46f8-8eb1-125c76420824","Type":"ContainerDied","Data":"be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d"}
Jul 28 18:07:37.085941 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.085876    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l" event={"ID":"e513629d-958d-46f8-8eb1-125c76420824","Type":"ContainerDied","Data":"b3afb9c5941c07438684ce3f37b1ce906b249e865d6e75fe58b2e2770584c9d4"}
Jul 28 18:07:37.085941 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.085907    2575 scope.go:117] "RemoveContainer" containerID="be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d"
Jul 28 18:07:37.095100 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.095082    2575 scope.go:117] "RemoveContainer" containerID="be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d"
Jul 28 18:07:37.095430 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:07:37.095411    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d\": container with ID starting with be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d not found: ID does not exist" containerID="be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d"
Jul 28 18:07:37.095481 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.095441    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d"} err="failed to get container status \"be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d\": rpc error: code = NotFound desc = could not find container \"be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d\": container with ID starting with be00c795d0d13ebe0e4b51a2c9ac12bb6d9d4549c6c420ff2db32784fb85c36d not found: ID does not exist"
Jul 28 18:07:37.107736 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.107704    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"]
Jul 28 18:07:37.113834 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.113803    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7qg8l"]
Jul 28 18:07:37.159128 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.159091    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fzjk\" (UniqueName: \"kubernetes.io/projected/e513629d-958d-46f8-8eb1-125c76420824-kube-api-access-5fzjk\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:07:37.159128 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:37.159121    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e513629d-958d-46f8-8eb1-125c76420824-extensions-socket-volume\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:07:38.400425 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:38.400385    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e513629d-958d-46f8-8eb1-125c76420824" path="/var/lib/kubelet/pods/e513629d-958d-46f8-8eb1-125c76420824/volumes"
Jul 28 18:07:52.470100 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.470070    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"]
Jul 28 18:07:52.470509 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.470399    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e513629d-958d-46f8-8eb1-125c76420824" containerName="manager"
Jul 28 18:07:52.470509 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.470409    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e513629d-958d-46f8-8eb1-125c76420824" containerName="manager"
Jul 28 18:07:52.470509 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.470463    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e513629d-958d-46f8-8eb1-125c76420824" containerName="manager"
Jul 28 18:07:52.472310 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.472295    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.476448 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.476417    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vqm97\""
Jul 28 18:07:52.488816 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.488791    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"]
Jul 28 18:07:52.602571 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602533    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.602571 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602578    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.602903 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.602903 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602659    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sgh7\" (UniqueName: \"kubernetes.io/projected/72b5e147-220c-4ff1-a739-bb9bf33d95b9-kube-api-access-7sgh7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.602903 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.602903 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.602903 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.602903 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602834    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.603221 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.602916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704013 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.703968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sgh7\" (UniqueName: \"kubernetes.io/projected/72b5e147-220c-4ff1-a739-bb9bf33d95b9-kube-api-access-7sgh7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704176    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704202    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704470    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704496    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704746 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704723    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.704803 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.704770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.705360 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.705330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.707034 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.707006    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.707203 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.707181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.712646 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.712617    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72b5e147-220c-4ff1-a739-bb9bf33d95b9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.712791 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.712672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sgh7\" (UniqueName: \"kubernetes.io/projected/72b5e147-220c-4ff1-a739-bb9bf33d95b9-kube-api-access-7sgh7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jbnxs\" (UID: \"72b5e147-220c-4ff1-a739-bb9bf33d95b9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:52.789020 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:52.788988    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:53.063839 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.063757    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"]
Jul 28 18:07:53.067272 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:07:53.067236    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b5e147_220c_4ff1_a739_bb9bf33d95b9.slice/crio-a2915ef5435a33c5ad670db62a2b9697c8ec271edc1da9a7ffae84e2eb455217 WatchSource:0}: Error finding container a2915ef5435a33c5ad670db62a2b9697c8ec271edc1da9a7ffae84e2eb455217: Status 404 returned error can't find the container with id a2915ef5435a33c5ad670db62a2b9697c8ec271edc1da9a7ffae84e2eb455217
Jul 28 18:07:53.069414 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.069381    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:07:53.069497 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.069448    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:07:53.069497 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.069490    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:07:53.163121 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.163079    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" event={"ID":"72b5e147-220c-4ff1-a739-bb9bf33d95b9","Type":"ContainerStarted","Data":"124312ecc4d7471fb643ae6613bf031be2ccf95665ccb4ff1133828b20563c2f"}
Jul 28 18:07:53.163317 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.163137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" event={"ID":"72b5e147-220c-4ff1-a739-bb9bf33d95b9","Type":"ContainerStarted","Data":"a2915ef5435a33c5ad670db62a2b9697c8ec271edc1da9a7ffae84e2eb455217"}
Jul 28 18:07:53.183591 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.183517    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" podStartSLOduration=1.183496054 podStartE2EDuration="1.183496054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:53.181445795 +0000 UTC m=+647.326998771" watchObservedRunningTime="2026-07-28 18:07:53.183496054 +0000 UTC m=+647.329049030"
Jul 28 18:07:53.789709 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:53.789667    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:54.248724 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.248687    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kx72x"]
Jul 28 18:07:54.352313 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.352280    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kx72x"]
Jul 28 18:07:54.352475 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.352324    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kx72x"]
Jul 28 18:07:54.352475 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.352401    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.354923 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.354902    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 18:07:54.418363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.418324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4457cff-3516-41ce-a452-d2803dd6c12e-config-file\") pod \"limitador-limitador-64c94fd64f-kx72x\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.418598 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.418372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn2rv\" (UniqueName: \"kubernetes.io/projected/c4457cff-3516-41ce-a452-d2803dd6c12e-kube-api-access-bn2rv\") pod \"limitador-limitador-64c94fd64f-kx72x\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.519033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.518946    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4457cff-3516-41ce-a452-d2803dd6c12e-config-file\") pod \"limitador-limitador-64c94fd64f-kx72x\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.519033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.518983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2rv\" (UniqueName: \"kubernetes.io/projected/c4457cff-3516-41ce-a452-d2803dd6c12e-kube-api-access-bn2rv\") pod \"limitador-limitador-64c94fd64f-kx72x\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.519578 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.519557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4457cff-3516-41ce-a452-d2803dd6c12e-config-file\") pod \"limitador-limitador-64c94fd64f-kx72x\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.526784 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.526752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn2rv\" (UniqueName: \"kubernetes.io/projected/c4457cff-3516-41ce-a452-d2803dd6c12e-kube-api-access-bn2rv\") pod \"limitador-limitador-64c94fd64f-kx72x\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.585634 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.585599    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:07:54.663806 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.663770    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:07:54.673018 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.672993    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:07:54.673149 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.673035    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:07:54.673149 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.673105    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:54.720356 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.720316    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd689fd4-6e2e-463f-aa0f-cc52af802eb6-config-file\") pod \"limitador-limitador-54d6d988d5-p7jzr\" (UID: \"dd689fd4-6e2e-463f-aa0f-cc52af802eb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:54.720556 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.720367    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7j5k\" (UniqueName: \"kubernetes.io/projected/dd689fd4-6e2e-463f-aa0f-cc52af802eb6-kube-api-access-f7j5k\") pod \"limitador-limitador-54d6d988d5-p7jzr\" (UID: \"dd689fd4-6e2e-463f-aa0f-cc52af802eb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:54.795630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.795559    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:54.814336 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.814313    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kx72x"]
Jul 28 18:07:54.816561 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:07:54.816535    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4457cff_3516_41ce_a452_d2803dd6c12e.slice/crio-af6cbc189cbcc4b59c9200130d0d1027c89b3f4d9a3c19289f832b6c4e10eee6 WatchSource:0}: Error finding container af6cbc189cbcc4b59c9200130d0d1027c89b3f4d9a3c19289f832b6c4e10eee6: Status 404 returned error can't find the container with id af6cbc189cbcc4b59c9200130d0d1027c89b3f4d9a3c19289f832b6c4e10eee6
Jul 28 18:07:54.820793 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.820773    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd689fd4-6e2e-463f-aa0f-cc52af802eb6-config-file\") pod \"limitador-limitador-54d6d988d5-p7jzr\" (UID: \"dd689fd4-6e2e-463f-aa0f-cc52af802eb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:54.820907 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.820819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j5k\" (UniqueName: \"kubernetes.io/projected/dd689fd4-6e2e-463f-aa0f-cc52af802eb6-kube-api-access-f7j5k\") pod \"limitador-limitador-54d6d988d5-p7jzr\" (UID: \"dd689fd4-6e2e-463f-aa0f-cc52af802eb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:54.821456 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.821436    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd689fd4-6e2e-463f-aa0f-cc52af802eb6-config-file\") pod \"limitador-limitador-54d6d988d5-p7jzr\" (UID: \"dd689fd4-6e2e-463f-aa0f-cc52af802eb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:54.830032 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.830010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7j5k\" (UniqueName: \"kubernetes.io/projected/dd689fd4-6e2e-463f-aa0f-cc52af802eb6-kube-api-access-f7j5k\") pod \"limitador-limitador-54d6d988d5-p7jzr\" (UID: \"dd689fd4-6e2e-463f-aa0f-cc52af802eb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:54.985952 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:54.985920    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:07:55.126986 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.126809    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:07:55.129731 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:07:55.129701    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd689fd4_6e2e_463f_aa0f_cc52af802eb6.slice/crio-58afc1eb2af6b816480a9b575558089cf0438f9465c7458b1b0b5d9335241be6 WatchSource:0}: Error finding container 58afc1eb2af6b816480a9b575558089cf0438f9465c7458b1b0b5d9335241be6: Status 404 returned error can't find the container with id 58afc1eb2af6b816480a9b575558089cf0438f9465c7458b1b0b5d9335241be6
Jul 28 18:07:55.176597 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.176556    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr" event={"ID":"dd689fd4-6e2e-463f-aa0f-cc52af802eb6","Type":"ContainerStarted","Data":"58afc1eb2af6b816480a9b575558089cf0438f9465c7458b1b0b5d9335241be6"}
Jul 28 18:07:55.177738 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.177717    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x" event={"ID":"c4457cff-3516-41ce-a452-d2803dd6c12e","Type":"ContainerStarted","Data":"af6cbc189cbcc4b59c9200130d0d1027c89b3f4d9a3c19289f832b6c4e10eee6"}
Jul 28 18:07:55.178062 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.178048    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:55.179181 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.179157    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:07:55.215701 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.215660    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvj5t"]
Jul 28 18:07:55.220735 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.220707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvj5t"
Jul 28 18:07:55.224782 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.223423    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ngqrn\""
Jul 28 18:07:55.224782 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.223635    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5zx2\" (UniqueName: \"kubernetes.io/projected/2207d036-26a1-46ac-8d31-197d1b9ab536-kube-api-access-q5zx2\") pod \"authorino-75d556bd77-jvj5t\" (UID: \"2207d036-26a1-46ac-8d31-197d1b9ab536\") " pod="kuadrant-system/authorino-75d556bd77-jvj5t"
Jul 28 18:07:55.227204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.227175    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvj5t"]
Jul 28 18:07:55.325001 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.324916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5zx2\" (UniqueName: \"kubernetes.io/projected/2207d036-26a1-46ac-8d31-197d1b9ab536-kube-api-access-q5zx2\") pod \"authorino-75d556bd77-jvj5t\" (UID: \"2207d036-26a1-46ac-8d31-197d1b9ab536\") " pod="kuadrant-system/authorino-75d556bd77-jvj5t"
Jul 28 18:07:55.340453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.340429    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5zx2\" (UniqueName: \"kubernetes.io/projected/2207d036-26a1-46ac-8d31-197d1b9ab536-kube-api-access-q5zx2\") pod \"authorino-75d556bd77-jvj5t\" (UID: \"2207d036-26a1-46ac-8d31-197d1b9ab536\") " pod="kuadrant-system/authorino-75d556bd77-jvj5t"
Jul 28 18:07:55.532773 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.532729    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvj5t"
Jul 28 18:07:55.706412 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:55.706377    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvj5t"]
Jul 28 18:07:55.712597 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:07:55.712561    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2207d036_26a1_46ac_8d31_197d1b9ab536.slice/crio-d202056c65608a08b14f48503720340ad3056ba26b07968567afaf5e18e5e14c WatchSource:0}: Error finding container d202056c65608a08b14f48503720340ad3056ba26b07968567afaf5e18e5e14c: Status 404 returned error can't find the container with id d202056c65608a08b14f48503720340ad3056ba26b07968567afaf5e18e5e14c
Jul 28 18:07:56.187943 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:07:56.187872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvj5t" event={"ID":"2207d036-26a1-46ac-8d31-197d1b9ab536","Type":"ContainerStarted","Data":"d202056c65608a08b14f48503720340ad3056ba26b07968567afaf5e18e5e14c"}
Jul 28 18:08:00.216107 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.216009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr" event={"ID":"dd689fd4-6e2e-463f-aa0f-cc52af802eb6","Type":"ContainerStarted","Data":"5701e38dd765633d300d1cee88d5a365de587a253aab58949c00eab850d3a68c"}
Jul 28 18:08:00.216547 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.216140    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:08:00.217733 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.217705    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x" event={"ID":"c4457cff-3516-41ce-a452-d2803dd6c12e","Type":"ContainerStarted","Data":"b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0"}
Jul 28 18:08:00.217887 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.217770    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:08:00.219327 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.219305    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvj5t" event={"ID":"2207d036-26a1-46ac-8d31-197d1b9ab536","Type":"ContainerStarted","Data":"9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a"}
Jul 28 18:08:00.237549 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.237501    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr" podStartSLOduration=1.3858744729999999 podStartE2EDuration="6.237484695s" totalImagesPullingTime="4.851610222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:00.235673568 +0000 UTC m=+654.381226543" watchObservedRunningTime="2026-07-28 18:08:00.237484695 +0000 UTC m=+654.383037684"
Jul 28 18:08:00.252807 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.252738    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x" podStartSLOduration=1.097535349 podStartE2EDuration="6.252718982s" totalImagesPullingTime="5.155183633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:00.252571324 +0000 UTC m=+654.398124299" watchObservedRunningTime="2026-07-28 18:08:00.252718982 +0000 UTC m=+654.398271956"
Jul 28 18:08:00.272737 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:00.272686    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-jvj5t" podStartSLOduration=1.029956378 podStartE2EDuration="5.272670306s" totalImagesPullingTime="4.242713928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:00.270778303 +0000 UTC m=+654.416331274" watchObservedRunningTime="2026-07-28 18:08:00.272670306 +0000 UTC m=+654.418223280"
Jul 28 18:08:11.226489 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.226457    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"
Jul 28 18:08:11.226954 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.226525    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:08:11.292535 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.289052    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kx72x"]
Jul 28 18:08:11.292535 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.289376    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x" podUID="c4457cff-3516-41ce-a452-d2803dd6c12e" containerName="limitador" containerID="cri-o://b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0" gracePeriod=30
Jul 28 18:08:11.848957 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.848928    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:08:11.971760 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.971668    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn2rv\" (UniqueName: \"kubernetes.io/projected/c4457cff-3516-41ce-a452-d2803dd6c12e-kube-api-access-bn2rv\") pod \"c4457cff-3516-41ce-a452-d2803dd6c12e\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") "
Jul 28 18:08:11.971760 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.971719    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4457cff-3516-41ce-a452-d2803dd6c12e-config-file\") pod \"c4457cff-3516-41ce-a452-d2803dd6c12e\" (UID: \"c4457cff-3516-41ce-a452-d2803dd6c12e\") "
Jul 28 18:08:11.972146 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.972125    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4457cff-3516-41ce-a452-d2803dd6c12e-config-file" (OuterVolumeSpecName: "config-file") pod "c4457cff-3516-41ce-a452-d2803dd6c12e" (UID: "c4457cff-3516-41ce-a452-d2803dd6c12e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:08:11.973898 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:11.973870    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4457cff-3516-41ce-a452-d2803dd6c12e-kube-api-access-bn2rv" (OuterVolumeSpecName: "kube-api-access-bn2rv") pod "c4457cff-3516-41ce-a452-d2803dd6c12e" (UID: "c4457cff-3516-41ce-a452-d2803dd6c12e"). InnerVolumeSpecName "kube-api-access-bn2rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:08:12.072777 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.072720    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn2rv\" (UniqueName: \"kubernetes.io/projected/c4457cff-3516-41ce-a452-d2803dd6c12e-kube-api-access-bn2rv\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:08:12.072777 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.072767    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4457cff-3516-41ce-a452-d2803dd6c12e-config-file\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:08:12.278741 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.278701    2575 generic.go:356] "Generic (PLEG): container finished" podID="c4457cff-3516-41ce-a452-d2803dd6c12e" containerID="b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0" exitCode=0
Jul 28 18:08:12.279227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.278750    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x"
Jul 28 18:08:12.279227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.278792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x" event={"ID":"c4457cff-3516-41ce-a452-d2803dd6c12e","Type":"ContainerDied","Data":"b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0"}
Jul 28 18:08:12.279227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.278885    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kx72x" event={"ID":"c4457cff-3516-41ce-a452-d2803dd6c12e","Type":"ContainerDied","Data":"af6cbc189cbcc4b59c9200130d0d1027c89b3f4d9a3c19289f832b6c4e10eee6"}
Jul 28 18:08:12.279227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.278923    2575 scope.go:117] "RemoveContainer" containerID="b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0"
Jul 28 18:08:12.288168 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.288151    2575 scope.go:117] "RemoveContainer" containerID="b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0"
Jul 28 18:08:12.288498 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:08:12.288479    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0\": container with ID starting with b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0 not found: ID does not exist" containerID="b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0"
Jul 28 18:08:12.288536 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.288509    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0"} err="failed to get container status \"b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0\": rpc error: code = NotFound desc = could not find container \"b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0\": container with ID starting with b9d6e9c6c031d42d39ff4f79266b6dc4ac1ab83f9afecf230f43caea37ec44b0 not found: ID does not exist"
Jul 28 18:08:12.301874 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.301828    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kx72x"]
Jul 28 18:08:12.305732 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.305705    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kx72x"]
Jul 28 18:08:12.400730 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:12.400697    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4457cff-3516-41ce-a452-d2803dd6c12e" path="/var/lib/kubelet/pods/c4457cff-3516-41ce-a452-d2803dd6c12e/volumes"
Jul 28 18:08:16.010994 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.010950    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"]
Jul 28 18:08:16.011428 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.011290    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4457cff-3516-41ce-a452-d2803dd6c12e" containerName="limitador"
Jul 28 18:08:16.011428 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.011300    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4457cff-3516-41ce-a452-d2803dd6c12e" containerName="limitador"
Jul 28 18:08:16.011428 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.011384    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4457cff-3516-41ce-a452-d2803dd6c12e" containerName="limitador"
Jul 28 18:08:16.014144 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.014116    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.016864 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.016828    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 18:08:16.017012 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.016827    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-42rk9\""
Jul 28 18:08:16.017652 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.017634    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 18:08:16.017737 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.017641    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 18:08:16.023802 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.023779    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"]
Jul 28 18:08:16.109244 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.109206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkpf8\" (UniqueName: \"kubernetes.io/projected/82317128-bc81-4a3c-a4f1-9426f3420643-kube-api-access-kkpf8\") pod \"postgres-69fb4594cf-rskz9\" (UID: \"82317128-bc81-4a3c-a4f1-9426f3420643\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.109398 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.109257    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/82317128-bc81-4a3c-a4f1-9426f3420643-data\") pod \"postgres-69fb4594cf-rskz9\" (UID: \"82317128-bc81-4a3c-a4f1-9426f3420643\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.210521 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.210478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkpf8\" (UniqueName: \"kubernetes.io/projected/82317128-bc81-4a3c-a4f1-9426f3420643-kube-api-access-kkpf8\") pod \"postgres-69fb4594cf-rskz9\" (UID: \"82317128-bc81-4a3c-a4f1-9426f3420643\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.210700 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.210533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/82317128-bc81-4a3c-a4f1-9426f3420643-data\") pod \"postgres-69fb4594cf-rskz9\" (UID: \"82317128-bc81-4a3c-a4f1-9426f3420643\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.210926 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.210911    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/82317128-bc81-4a3c-a4f1-9426f3420643-data\") pod \"postgres-69fb4594cf-rskz9\" (UID: \"82317128-bc81-4a3c-a4f1-9426f3420643\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.229658 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.229637    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkpf8\" (UniqueName: \"kubernetes.io/projected/82317128-bc81-4a3c-a4f1-9426f3420643-kube-api-access-kkpf8\") pod \"postgres-69fb4594cf-rskz9\" (UID: \"82317128-bc81-4a3c-a4f1-9426f3420643\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.326161 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.326055    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:16.466042 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:16.466015    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"]
Jul 28 18:08:16.468689 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:08:16.468661    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82317128_bc81_4a3c_a4f1_9426f3420643.slice/crio-95bc611e13638dec4dab386629c7f8542240ec10c043e34872ca088199a49970 WatchSource:0}: Error finding container 95bc611e13638dec4dab386629c7f8542240ec10c043e34872ca088199a49970: Status 404 returned error can't find the container with id 95bc611e13638dec4dab386629c7f8542240ec10c043e34872ca088199a49970
Jul 28 18:08:17.304057 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:17.304021    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9" event={"ID":"82317128-bc81-4a3c-a4f1-9426f3420643","Type":"ContainerStarted","Data":"95bc611e13638dec4dab386629c7f8542240ec10c043e34872ca088199a49970"}
Jul 28 18:08:22.331929 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:22.331892    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9" event={"ID":"82317128-bc81-4a3c-a4f1-9426f3420643","Type":"ContainerStarted","Data":"7ee574aa1856608e394b27d40205ded441b9a5e2eac4a2c28bc8ea28dfa91d32"}
Jul 28 18:08:22.332363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:22.332083    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:22.349262 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:22.349205    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9" podStartSLOduration=2.290593641 podStartE2EDuration="7.349186034s" totalImagesPullingTime="5.058592393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:22.347541841 +0000 UTC m=+676.493094815" watchObservedRunningTime="2026-07-28 18:08:22.349186034 +0000 UTC m=+676.494739008"
Jul 28 18:08:28.366993 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:28.366962    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rskz9"
Jul 28 18:08:29.333218 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:29.333182    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-696d56d974-gdn95"]
Jul 28 18:08:29.339880 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:29.339825    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696d56d974-gdn95"
Jul 28 18:08:29.349021 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:29.348982    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-696d56d974-gdn95"]
Jul 28 18:08:29.437148 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:29.437101    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67cz7\" (UniqueName: \"kubernetes.io/projected/70936f92-899f-4636-973f-31e22104664c-kube-api-access-67cz7\") pod \"authorino-696d56d974-gdn95\" (UID: \"70936f92-899f-4636-973f-31e22104664c\") " pod="kuadrant-system/authorino-696d56d974-gdn95"
Jul 28 18:08:29.478146 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:29.478109    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-696d56d974-gdn95"]
Jul 28 18:08:29.478431 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:08:29.478407    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-67cz7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-696d56d974-gdn95" podUID="70936f92-899f-4636-973f-31e22104664c"
Jul 28 18:08:29.538359 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:29.538322    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67cz7\" (UniqueName: \"kubernetes.io/projected/70936f92-899f-4636-973f-31e22104664c-kube-api-access-67cz7\") pod \"authorino-696d56d974-gdn95\" (UID: \"70936f92-899f-4636-973f-31e22104664c\") " pod="kuadrant-system/authorino-696d56d974-gdn95"
Jul 28 18:08:29.548477 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:29.548440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67cz7\" (UniqueName: \"kubernetes.io/projected/70936f92-899f-4636-973f-31e22104664c-kube-api-access-67cz7\") pod \"authorino-696d56d974-gdn95\" (UID: \"70936f92-899f-4636-973f-31e22104664c\") " pod="kuadrant-system/authorino-696d56d974-gdn95"
Jul 28 18:08:30.374551 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.374518    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696d56d974-gdn95"
Jul 28 18:08:30.381042 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.381014    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696d56d974-gdn95"
Jul 28 18:08:30.446105 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.446069    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67cz7\" (UniqueName: \"kubernetes.io/projected/70936f92-899f-4636-973f-31e22104664c-kube-api-access-67cz7\") pod \"70936f92-899f-4636-973f-31e22104664c\" (UID: \"70936f92-899f-4636-973f-31e22104664c\") "
Jul 28 18:08:30.448230 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.448196    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70936f92-899f-4636-973f-31e22104664c-kube-api-access-67cz7" (OuterVolumeSpecName: "kube-api-access-67cz7") pod "70936f92-899f-4636-973f-31e22104664c" (UID: "70936f92-899f-4636-973f-31e22104664c"). InnerVolumeSpecName "kube-api-access-67cz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:08:30.530338 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.530296    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvj5t"]
Jul 28 18:08:30.530519 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.530495    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-jvj5t" podUID="2207d036-26a1-46ac-8d31-197d1b9ab536" containerName="authorino" containerID="cri-o://9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a" gracePeriod=30
Jul 28 18:08:30.547072 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.547046    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67cz7\" (UniqueName: \"kubernetes.io/projected/70936f92-899f-4636-973f-31e22104664c-kube-api-access-67cz7\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:08:30.791748 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.791714    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvj5t"
Jul 28 18:08:30.950845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.950766    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5zx2\" (UniqueName: \"kubernetes.io/projected/2207d036-26a1-46ac-8d31-197d1b9ab536-kube-api-access-q5zx2\") pod \"2207d036-26a1-46ac-8d31-197d1b9ab536\" (UID: \"2207d036-26a1-46ac-8d31-197d1b9ab536\") "
Jul 28 18:08:30.953121 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:30.953082    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2207d036-26a1-46ac-8d31-197d1b9ab536-kube-api-access-q5zx2" (OuterVolumeSpecName: "kube-api-access-q5zx2") pod "2207d036-26a1-46ac-8d31-197d1b9ab536" (UID: "2207d036-26a1-46ac-8d31-197d1b9ab536"). InnerVolumeSpecName "kube-api-access-q5zx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:08:31.051684 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.051647    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5zx2\" (UniqueName: \"kubernetes.io/projected/2207d036-26a1-46ac-8d31-197d1b9ab536-kube-api-access-q5zx2\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:08:31.380217 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.380178    2575 generic.go:356] "Generic (PLEG): container finished" podID="2207d036-26a1-46ac-8d31-197d1b9ab536" containerID="9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a" exitCode=0
Jul 28 18:08:31.380217 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.380220    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvj5t"
Jul 28 18:08:31.380448 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.380267    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvj5t" event={"ID":"2207d036-26a1-46ac-8d31-197d1b9ab536","Type":"ContainerDied","Data":"9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a"}
Jul 28 18:08:31.380448 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.380314    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvj5t" event={"ID":"2207d036-26a1-46ac-8d31-197d1b9ab536","Type":"ContainerDied","Data":"d202056c65608a08b14f48503720340ad3056ba26b07968567afaf5e18e5e14c"}
Jul 28 18:08:31.380448 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.380332    2575 scope.go:117] "RemoveContainer" containerID="9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a"
Jul 28 18:08:31.380600 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.380465    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696d56d974-gdn95"
Jul 28 18:08:31.390633 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.390606    2575 scope.go:117] "RemoveContainer" containerID="9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a"
Jul 28 18:08:31.390959 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:08:31.390941    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a\": container with ID starting with 9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a not found: ID does not exist" containerID="9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a"
Jul 28 18:08:31.391040 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.390967    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a"} err="failed to get container status \"9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a\": rpc error: code = NotFound desc = could not find container \"9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a\": container with ID starting with 9586e9e6e677b31cd74c381d56d9f8d0a6511da352b2460e7fc124892fa5f71a not found: ID does not exist"
Jul 28 18:08:31.429205 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.429171    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvj5t"]
Jul 28 18:08:31.440061 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.440028    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvj5t"]
Jul 28 18:08:31.472027 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.471984    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-696d56d974-gdn95"]
Jul 28 18:08:31.477955 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:31.477927    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-696d56d974-gdn95"]
Jul 28 18:08:32.400523 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:32.400491    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2207d036-26a1-46ac-8d31-197d1b9ab536" path="/var/lib/kubelet/pods/2207d036-26a1-46ac-8d31-197d1b9ab536/volumes"
Jul 28 18:08:32.400834 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:32.400821    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70936f92-899f-4636-973f-31e22104664c" path="/var/lib/kubelet/pods/70936f92-899f-4636-973f-31e22104664c/volumes"
Jul 28 18:08:35.618941 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.618906    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5655468897-wmlvr"]
Jul 28 18:08:35.619354 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.619240    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2207d036-26a1-46ac-8d31-197d1b9ab536" containerName="authorino"
Jul 28 18:08:35.619354 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.619250    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2207d036-26a1-46ac-8d31-197d1b9ab536" containerName="authorino"
Jul 28 18:08:35.619354 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.619311    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2207d036-26a1-46ac-8d31-197d1b9ab536" containerName="authorino"
Jul 28 18:08:35.844634 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.844588    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5655468897-wmlvr"]
Jul 28 18:08:35.844824 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.844713    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:35.848642 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.848616    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 18:08:35.848642 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.848628    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 18:08:35.848845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.848710    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-c4ndl\""
Jul 28 18:08:35.999124 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.999088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd2632be-e92b-4584-a349-852464063060-webhook-cert\") pod \"maas-controller-5655468897-wmlvr\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") " pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:35.999292 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:35.999208    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2lmd\" (UniqueName: \"kubernetes.io/projected/fd2632be-e92b-4584-a349-852464063060-kube-api-access-l2lmd\") pod \"maas-controller-5655468897-wmlvr\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") " pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:36.099648 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:36.099606    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2lmd\" (UniqueName: \"kubernetes.io/projected/fd2632be-e92b-4584-a349-852464063060-kube-api-access-l2lmd\") pod \"maas-controller-5655468897-wmlvr\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") " pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:36.099848 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:36.099693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd2632be-e92b-4584-a349-852464063060-webhook-cert\") pod \"maas-controller-5655468897-wmlvr\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") " pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:36.102291 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:36.102264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd2632be-e92b-4584-a349-852464063060-webhook-cert\") pod \"maas-controller-5655468897-wmlvr\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") " pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:36.110387 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:36.110362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2lmd\" (UniqueName: \"kubernetes.io/projected/fd2632be-e92b-4584-a349-852464063060-kube-api-access-l2lmd\") pod \"maas-controller-5655468897-wmlvr\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") " pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:36.154693 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:36.154654    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:36.312226 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:36.312198    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5655468897-wmlvr"]
Jul 28 18:08:36.315103 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:08:36.315065    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2632be_e92b_4584_a349_852464063060.slice/crio-23244558660abcfc72dab8fa565e90ef95902614c715732232d64e732b0a377d WatchSource:0}: Error finding container 23244558660abcfc72dab8fa565e90ef95902614c715732232d64e732b0a377d: Status 404 returned error can't find the container with id 23244558660abcfc72dab8fa565e90ef95902614c715732232d64e732b0a377d
Jul 28 18:08:36.406016 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:36.405983    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-wmlvr" event={"ID":"fd2632be-e92b-4584-a349-852464063060","Type":"ContainerStarted","Data":"23244558660abcfc72dab8fa565e90ef95902614c715732232d64e732b0a377d"}
Jul 28 18:08:39.423586 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:39.423550    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-wmlvr" event={"ID":"fd2632be-e92b-4584-a349-852464063060","Type":"ContainerStarted","Data":"134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69"}
Jul 28 18:08:39.423989 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:39.423618    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:50.430495 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:50.430464    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:08:50.450119 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:08:50.450066    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5655468897-wmlvr" podStartSLOduration=13.279351287 podStartE2EDuration="15.450048216s" totalImagesPullingTime="2.170696929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:39.459597657 +0000 UTC m=+693.605150631" watchObservedRunningTime="2026-07-28 18:08:50.450048216 +0000 UTC m=+704.595601190"
Jul 28 18:09:12.206680 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:12.206626    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:09:19.620298 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:19.620211    2575 generic.go:356] "Generic (PLEG): container finished" podID="72b5e147-220c-4ff1-a739-bb9bf33d95b9" containerID="124312ecc4d7471fb643ae6613bf031be2ccf95665ccb4ff1133828b20563c2f" exitCode=0
Jul 28 18:09:19.620298 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:19.620283    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" event={"ID":"72b5e147-220c-4ff1-a739-bb9bf33d95b9","Type":"ContainerDied","Data":"124312ecc4d7471fb643ae6613bf031be2ccf95665ccb4ff1133828b20563c2f"}
Jul 28 18:09:19.620761 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:19.620742    2575 scope.go:117] "RemoveContainer" containerID="124312ecc4d7471fb643ae6613bf031be2ccf95665ccb4ff1133828b20563c2f"
Jul 28 18:09:19.621769 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:19.621744    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:09:19.621910 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:19.621813    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:09:19.621910 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:19.621845    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:09:20.626709 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:20.626668    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" event={"ID":"72b5e147-220c-4ff1-a739-bb9bf33d95b9","Type":"ContainerStarted","Data":"a1015812d5cebc7db4863e8659c2a146f0f9c5b77373361ef18c874c6b315158"}
Jul 28 18:09:20.789822 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:20.789784    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:09:20.791033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:20.791005    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jbnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 28 18:09:20.791170 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:20.791081    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" podUID="72b5e147-220c-4ff1-a739-bb9bf33d95b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 28 18:09:21.789728 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:21.789689    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jbnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 28 18:09:21.790165 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:21.789749    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" podUID="72b5e147-220c-4ff1-a739-bb9bf33d95b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 28 18:09:22.789343 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:22.789304    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jbnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 28 18:09:22.789529 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:22.789359    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" podUID="72b5e147-220c-4ff1-a739-bb9bf33d95b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 28 18:09:23.790171 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:23.790131    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jbnxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 28 18:09:23.790574 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:23.790188    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs" podUID="72b5e147-220c-4ff1-a739-bb9bf33d95b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 28 18:09:24.794171 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:24.794141    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:09:24.794573 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:24.794483    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:09:24.795342 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:24.795316    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jbnxs"
Jul 28 18:09:37.885190 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:37.885152    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:09:58.986431 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:58.986398    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"]
Jul 28 18:09:58.989111 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:58.989089    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:58.994091 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:58.994071    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 18:09:58.995128 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:58.995107    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jk58p\""
Jul 28 18:09:58.995212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:58.995127    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 18:09:58.995212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:58.995107    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 18:09:59.007261 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.007238    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"]
Jul 28 18:09:59.052195 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.052160    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.052370 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.052221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.052370 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.052239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.052370 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.052255    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.052370 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.052278    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.052370 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.052299    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqnx\" (UniqueName: \"kubernetes.io/projected/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-kube-api-access-pfqnx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153032 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.152997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153237 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153237 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153237 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153212    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153410 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153410 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153284    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqnx\" (UniqueName: \"kubernetes.io/projected/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-kube-api-access-pfqnx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153410 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153389    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153608 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153584    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.153666 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.153592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.155336 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.155310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.155587 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.155567    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.162006 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.161976    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqnx\" (UniqueName: \"kubernetes.io/projected/3675f496-9aae-46b1-b3f9-5b4233ba7eb4-kube-api-access-pfqnx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fhtrr\" (UID: \"3675f496-9aae-46b1-b3f9-5b4233ba7eb4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.300678 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.300640    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:09:59.434753 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.434724    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"]
Jul 28 18:09:59.436755 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:09:59.436720    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3675f496_9aae_46b1_b3f9_5b4233ba7eb4.slice/crio-f5e3c6f221f5afdc1a645944d4a271e49bcfaedefa912170766f6daf3265057f WatchSource:0}: Error finding container f5e3c6f221f5afdc1a645944d4a271e49bcfaedefa912170766f6daf3265057f: Status 404 returned error can't find the container with id f5e3c6f221f5afdc1a645944d4a271e49bcfaedefa912170766f6daf3265057f
Jul 28 18:09:59.438351 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.438327    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:09:59.792517 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.792479    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:09:59.827282 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:09:59.827246    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr" event={"ID":"3675f496-9aae-46b1-b3f9-5b4233ba7eb4","Type":"ContainerStarted","Data":"f5e3c6f221f5afdc1a645944d4a271e49bcfaedefa912170766f6daf3265057f"}
Jul 28 18:10:00.835497 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:00.835451    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr" event={"ID":"3675f496-9aae-46b1-b3f9-5b4233ba7eb4","Type":"ContainerStarted","Data":"03ff4bccc3b5fb071a4996b746502af0beef69c1905627deb14d32a33cc61849"}
Jul 28 18:10:00.855122 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:00.855067    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr" podStartSLOduration=1.808298639 podStartE2EDuration="2.855048626s" totalImagesPullingTime="1.046749987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:10:00.854600563 +0000 UTC m=+775.000153538" watchObservedRunningTime="2026-07-28 18:10:00.855048626 +0000 UTC m=+775.000601600"
Jul 28 18:10:09.301557 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:09.301506    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:10:09.302095 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:09.301585    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:10:09.314201 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:09.314173    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:10:09.888501 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:09.888469    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fhtrr"
Jul 28 18:10:10.487820 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.487782    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"]
Jul 28 18:10:10.491654 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.491624    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.494351 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.494331    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 18:10:10.503985 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.503952    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"]
Jul 28 18:10:10.658154 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.658107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.658154 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.658153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.658411 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.658249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.658411 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.658304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.658411 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.658352    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7gxc\" (UniqueName: \"kubernetes.io/projected/670efaf0-f405-4546-981a-539aa78dc5b0-kube-api-access-n7gxc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.658411 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.658399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/670efaf0-f405-4546-981a-539aa78dc5b0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.759733 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.759690    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.759733 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.759732    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.760033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.759789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.760033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.759818    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.760033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.759871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7gxc\" (UniqueName: \"kubernetes.io/projected/670efaf0-f405-4546-981a-539aa78dc5b0-kube-api-access-n7gxc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.760033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.759905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/670efaf0-f405-4546-981a-539aa78dc5b0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.760246 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.760201    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.760246 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.760219    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.760348 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.760294    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.762277 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.762247    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/670efaf0-f405-4546-981a-539aa78dc5b0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.762569 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.762549    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/670efaf0-f405-4546-981a-539aa78dc5b0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.771147 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.771116    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7gxc\" (UniqueName: \"kubernetes.io/projected/670efaf0-f405-4546-981a-539aa78dc5b0-kube-api-access-n7gxc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rfrsv\" (UID: \"670efaf0-f405-4546-981a-539aa78dc5b0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.804297 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.804257    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:10.961291 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:10.961255    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"]
Jul 28 18:10:10.966222 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:10:10.966191    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod670efaf0_f405_4546_981a_539aa78dc5b0.slice/crio-7823ec3be5c75dce589b1861d391fcce6add0286cf3ab76b597396b0e158ab9e WatchSource:0}: Error finding container 7823ec3be5c75dce589b1861d391fcce6add0286cf3ab76b597396b0e158ab9e: Status 404 returned error can't find the container with id 7823ec3be5c75dce589b1861d391fcce6add0286cf3ab76b597396b0e158ab9e
Jul 28 18:10:11.805835 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:11.805802    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:10:11.888309 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:11.888273    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv" event={"ID":"670efaf0-f405-4546-981a-539aa78dc5b0","Type":"ContainerStarted","Data":"73c07f4d780cbfb88fc84ee58296ead3c3a2797c85afc54b9e060bbb7f266227"}
Jul 28 18:10:11.888497 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:11.888339    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv" event={"ID":"670efaf0-f405-4546-981a-539aa78dc5b0","Type":"ContainerStarted","Data":"7823ec3be5c75dce589b1861d391fcce6add0286cf3ab76b597396b0e158ab9e"}
Jul 28 18:10:11.913000 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:11.912943    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv" podStartSLOduration=1.68830668 podStartE2EDuration="1.912923986s" totalImagesPullingTime="224.617306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:10:11.911304124 +0000 UTC m=+786.056857089" watchObservedRunningTime="2026-07-28 18:10:11.912923986 +0000 UTC m=+786.058476960"
Jul 28 18:10:19.198342 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:19.198308    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:10:20.805412 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:20.805369    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:20.805835 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:20.805442    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:20.818224 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:20.818192    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:20.944644 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:20.944617    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rfrsv"
Jul 28 18:10:30.385511 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.385423    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"]
Jul 28 18:10:30.404340 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.404306    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.407040 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.407016    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 18:10:30.412845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.412817    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"]
Jul 28 18:10:30.446232 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.446194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.446232 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.446237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.446457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.446255    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm5mm\" (UniqueName: \"kubernetes.io/projected/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-kube-api-access-zm5mm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.446457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.446361    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.446457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.446404    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.446557 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.446481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.547544 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.547505    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.547767 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.547563    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.547767 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.547597    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.547767 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.547657    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.547767 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.547686    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.547767 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.547716    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm5mm\" (UniqueName: \"kubernetes.io/projected/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-kube-api-access-zm5mm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.548154 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.548114    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.548238 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.548123    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.548331 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.548307    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.550280 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.550248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.550805 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.550774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.557036 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.557008    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm5mm\" (UniqueName: \"kubernetes.io/projected/fb9ffaec-fd0d-4355-9d11-de9cedea10f5-kube-api-access-zm5mm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8s8sn\" (UID: \"fb9ffaec-fd0d-4355-9d11-de9cedea10f5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.717195 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.717091    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:30.860884 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.860838    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"]
Jul 28 18:10:30.863172 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:10:30.863140    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9ffaec_fd0d_4355_9d11_de9cedea10f5.slice/crio-6a64040beb0fc8da4a95a225139d704447f9a04797223cb60885bc830da1b2f1 WatchSource:0}: Error finding container 6a64040beb0fc8da4a95a225139d704447f9a04797223cb60885bc830da1b2f1: Status 404 returned error can't find the container with id 6a64040beb0fc8da4a95a225139d704447f9a04797223cb60885bc830da1b2f1
Jul 28 18:10:30.983092 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:30.982990    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn" event={"ID":"fb9ffaec-fd0d-4355-9d11-de9cedea10f5","Type":"ContainerStarted","Data":"6a64040beb0fc8da4a95a225139d704447f9a04797223cb60885bc830da1b2f1"}
Jul 28 18:10:31.990183 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:31.990150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn" event={"ID":"fb9ffaec-fd0d-4355-9d11-de9cedea10f5","Type":"ContainerStarted","Data":"96277dac20b521fdeb651ac1a1e588762e90c7e4579414bf2ea5d372dab07d2f"}
Jul 28 18:10:32.010885 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:32.010813    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn" podStartSLOduration=1.787586955 podStartE2EDuration="2.010796138s" totalImagesPullingTime="223.209183ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:10:32.01046266 +0000 UTC m=+806.156015635" watchObservedRunningTime="2026-07-28 18:10:32.010796138 +0000 UTC m=+806.156349113"
Jul 28 18:10:33.194602 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:33.194566    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:10:35.580150 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.580118    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"]
Jul 28 18:10:35.582868 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.582826    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.585542 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.585521    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 18:10:35.594580 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.594552    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"]
Jul 28 18:10:35.698312 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.698277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.698312 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.698314    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29a68bcf-3064-4efc-87bc-36326283be16-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.698588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.698331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmdph\" (UniqueName: \"kubernetes.io/projected/29a68bcf-3064-4efc-87bc-36326283be16-kube-api-access-fmdph\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.698588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.698355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.698588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.698398    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.698588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.698425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800031 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.799971    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800031 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800033    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29a68bcf-3064-4efc-87bc-36326283be16-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800304 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmdph\" (UniqueName: \"kubernetes.io/projected/29a68bcf-3064-4efc-87bc-36326283be16-kube-api-access-fmdph\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800304 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800304 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800118    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800304 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800518    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800609 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800567    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.800609 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.800590    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.802471 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.802447    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a68bcf-3064-4efc-87bc-36326283be16-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.802731 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.802713    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29a68bcf-3064-4efc-87bc-36326283be16-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.810892 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.810833    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmdph\" (UniqueName: \"kubernetes.io/projected/29a68bcf-3064-4efc-87bc-36326283be16-kube-api-access-fmdph\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp\" (UID: \"29a68bcf-3064-4efc-87bc-36326283be16\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:35.892717 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:35.892623    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:36.038908 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:36.038813    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"]
Jul 28 18:10:36.041474 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:10:36.041439    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29a68bcf_3064_4efc_87bc_36326283be16.slice/crio-f3e740019aeafc249982a7127a2a8ba51e7c6ea10a4d173fe4cb77d42b084b77 WatchSource:0}: Error finding container f3e740019aeafc249982a7127a2a8ba51e7c6ea10a4d173fe4cb77d42b084b77: Status 404 returned error can't find the container with id f3e740019aeafc249982a7127a2a8ba51e7c6ea10a4d173fe4cb77d42b084b77
Jul 28 18:10:37.017379 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:37.017344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp" event={"ID":"29a68bcf-3064-4efc-87bc-36326283be16","Type":"ContainerStarted","Data":"dfcc44980f8210e6837b3ea2b34d7daa2a6e66f19e24912c4874b4c83b249882"}
Jul 28 18:10:37.017793 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:37.017395    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp" event={"ID":"29a68bcf-3064-4efc-87bc-36326283be16","Type":"ContainerStarted","Data":"f3e740019aeafc249982a7127a2a8ba51e7c6ea10a4d173fe4cb77d42b084b77"}
Jul 28 18:10:37.034657 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:37.034598    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp" podStartSLOduration=1.826464944 podStartE2EDuration="2.034581171s" totalImagesPullingTime="208.116227ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:10:37.033918546 +0000 UTC m=+811.179471521" watchObservedRunningTime="2026-07-28 18:10:37.034581171 +0000 UTC m=+811.180134162"
Jul 28 18:10:40.718264 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:40.718219    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:40.718898 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:40.718307    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:40.734840 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:40.734796    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:41.050258 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:41.050222    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8s8sn"
Jul 28 18:10:45.690052 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:45.690014    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:10:45.893235 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:45.893186    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:45.893235 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:45.893252    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:45.906632 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:45.906594    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:10:46.074061 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:10:46.074019    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp"
Jul 28 18:11:57.897645 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:11:57.897559    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:12:07.220189 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:07.220163    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:12:07.222289 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:07.222264    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:12:07.700308 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:07.700265    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:12:15.769372 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:15.769334    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5655468897-wmlvr"]
Jul 28 18:12:15.769910 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:15.769576    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5655468897-wmlvr" podUID="fd2632be-e92b-4584-a349-852464063060" containerName="manager" containerID="cri-o://134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69" gracePeriod=30
Jul 28 18:12:16.024351 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.024272    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:12:16.154049 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.154012    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2lmd\" (UniqueName: \"kubernetes.io/projected/fd2632be-e92b-4584-a349-852464063060-kube-api-access-l2lmd\") pod \"fd2632be-e92b-4584-a349-852464063060\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") "
Jul 28 18:12:16.154222 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.154145    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd2632be-e92b-4584-a349-852464063060-webhook-cert\") pod \"fd2632be-e92b-4584-a349-852464063060\" (UID: \"fd2632be-e92b-4584-a349-852464063060\") "
Jul 28 18:12:16.156414 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.156380    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2632be-e92b-4584-a349-852464063060-kube-api-access-l2lmd" (OuterVolumeSpecName: "kube-api-access-l2lmd") pod "fd2632be-e92b-4584-a349-852464063060" (UID: "fd2632be-e92b-4584-a349-852464063060"). InnerVolumeSpecName "kube-api-access-l2lmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:12:16.156414 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.156389    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd2632be-e92b-4584-a349-852464063060-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fd2632be-e92b-4584-a349-852464063060" (UID: "fd2632be-e92b-4584-a349-852464063060"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:12:16.255447 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.255398    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd2632be-e92b-4584-a349-852464063060-webhook-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:12:16.255447 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.255438    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2lmd\" (UniqueName: \"kubernetes.io/projected/fd2632be-e92b-4584-a349-852464063060-kube-api-access-l2lmd\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:12:16.513704 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.513670    2575 generic.go:356] "Generic (PLEG): container finished" podID="fd2632be-e92b-4584-a349-852464063060" containerID="134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69" exitCode=0
Jul 28 18:12:16.513969 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.513712    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-wmlvr"
Jul 28 18:12:16.513969 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.513744    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-wmlvr" event={"ID":"fd2632be-e92b-4584-a349-852464063060","Type":"ContainerDied","Data":"134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69"}
Jul 28 18:12:16.513969 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.513788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-wmlvr" event={"ID":"fd2632be-e92b-4584-a349-852464063060","Type":"ContainerDied","Data":"23244558660abcfc72dab8fa565e90ef95902614c715732232d64e732b0a377d"}
Jul 28 18:12:16.513969 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.513810    2575 scope.go:117] "RemoveContainer" containerID="134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69"
Jul 28 18:12:16.522899 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.522877    2575 scope.go:117] "RemoveContainer" containerID="134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69"
Jul 28 18:12:16.523230 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:12:16.523210    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69\": container with ID starting with 134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69 not found: ID does not exist" containerID="134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69"
Jul 28 18:12:16.523284 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.523242    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69"} err="failed to get container status \"134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69\": rpc error: code = NotFound desc = could not find container \"134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69\": container with ID starting with 134d496d8e18072cb98e187b80fb6b66971a8a63224376758fda826029132f69 not found: ID does not exist"
Jul 28 18:12:16.532413 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.532383    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5655468897-wmlvr"]
Jul 28 18:12:16.535800 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:16.535768    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5655468897-wmlvr"]
Jul 28 18:12:18.403510 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:18.400771    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd2632be-e92b-4584-a349-852464063060" path="/var/lib/kubelet/pods/fd2632be-e92b-4584-a349-852464063060/volumes"
Jul 28 18:12:19.098951 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:12:19.098906    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:13:58.505223 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:13:58.505182    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:14:10.104119 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:14:10.104083    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:14:46.297169 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:14:46.297127    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:14:56.315727 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:14:56.315686    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:15:07.397507 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:15:07.397413    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:15:15.388104 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:15:15.388060    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:15:26.813008 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:15:26.812963    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:15:36.563231 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:15:36.563191    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:15:47.897357 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:15:47.897320    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:15:57.891343 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:15:57.891305    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:16:20.414193 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:16:20.414159    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:16:32.094430 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:16:32.094333    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:16:54.699204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:16:54.699163    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:17:05.508156 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:17:05.508118    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:17:07.248102 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:17:07.248055    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:17:07.250999 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:17:07.250972    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:17:15.502958 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:17:15.502917    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:17:32.602962 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:17:32.602925    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:17:42.605710 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:17:42.605667    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:17:58.403710 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:17:58.403629    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:18:07.903005 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:07.902958    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:18:19.897086 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:19.897044    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:18:29.411495 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:29.411450    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:18:46.295026 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:46.294985    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"]
Jul 28 18:18:46.295526 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:46.295272    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm" podUID="dd950a52-7d1d-40d6-b78d-76393c222427" containerName="manager" containerID="cri-o://303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d" gracePeriod=10
Jul 28 18:18:49.058197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.058165    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:18:49.068301 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.068269    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd950a52-7d1d-40d6-b78d-76393c222427-extensions-socket-volume\") pod \"dd950a52-7d1d-40d6-b78d-76393c222427\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") "
Jul 28 18:18:49.068471 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.068324    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pf6l\" (UniqueName: \"kubernetes.io/projected/dd950a52-7d1d-40d6-b78d-76393c222427-kube-api-access-2pf6l\") pod \"dd950a52-7d1d-40d6-b78d-76393c222427\" (UID: \"dd950a52-7d1d-40d6-b78d-76393c222427\") "
Jul 28 18:18:49.068685 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.068662    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd950a52-7d1d-40d6-b78d-76393c222427-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dd950a52-7d1d-40d6-b78d-76393c222427" (UID: "dd950a52-7d1d-40d6-b78d-76393c222427"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:18:49.070439 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.070410    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd950a52-7d1d-40d6-b78d-76393c222427-kube-api-access-2pf6l" (OuterVolumeSpecName: "kube-api-access-2pf6l") pod "dd950a52-7d1d-40d6-b78d-76393c222427" (UID: "dd950a52-7d1d-40d6-b78d-76393c222427"). InnerVolumeSpecName "kube-api-access-2pf6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:18:49.169471 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.169383    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd950a52-7d1d-40d6-b78d-76393c222427-extensions-socket-volume\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:18:49.169471 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.169415    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pf6l\" (UniqueName: \"kubernetes.io/projected/dd950a52-7d1d-40d6-b78d-76393c222427-kube-api-access-2pf6l\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:18:49.447215 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.447124    2575 generic.go:356] "Generic (PLEG): container finished" podID="dd950a52-7d1d-40d6-b78d-76393c222427" containerID="303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d" exitCode=0
Jul 28 18:18:49.447215 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.447172    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"
Jul 28 18:18:49.447215 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.447198    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm" event={"ID":"dd950a52-7d1d-40d6-b78d-76393c222427","Type":"ContainerDied","Data":"303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d"}
Jul 28 18:18:49.447472 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.447248    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm" event={"ID":"dd950a52-7d1d-40d6-b78d-76393c222427","Type":"ContainerDied","Data":"567c213882d0ec72ab477155c702fbaca606cada721c21a5bd2536d74ce8fe1f"}
Jul 28 18:18:49.447472 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.447269    2575 scope.go:117] "RemoveContainer" containerID="303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d"
Jul 28 18:18:49.457377 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.457353    2575 scope.go:117] "RemoveContainer" containerID="303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d"
Jul 28 18:18:49.457692 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:18:49.457673    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d\": container with ID starting with 303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d not found: ID does not exist" containerID="303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d"
Jul 28 18:18:49.457733 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.457703    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d"} err="failed to get container status \"303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d\": rpc error: code = NotFound desc = could not find container \"303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d\": container with ID starting with 303424dce32249917377dc7a9cc7488c6326b939f5e681300486c1a1e2000e8d not found: ID does not exist"
Jul 28 18:18:49.474101 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.474073    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"]
Jul 28 18:18:49.479880 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:49.479839    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fpjqm"]
Jul 28 18:18:50.400815 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:18:50.400782    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd950a52-7d1d-40d6-b78d-76393c222427" path="/var/lib/kubelet/pods/dd950a52-7d1d-40d6-b78d-76393c222427/volumes"
Jul 28 18:19:47.459060 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.459024    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"]
Jul 28 18:19:47.460580 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.459454    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd2632be-e92b-4584-a349-852464063060" containerName="manager"
Jul 28 18:19:47.460580 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.459468    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd2632be-e92b-4584-a349-852464063060" containerName="manager"
Jul 28 18:19:47.460580 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.459489    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd950a52-7d1d-40d6-b78d-76393c222427" containerName="manager"
Jul 28 18:19:47.460580 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.459496    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd950a52-7d1d-40d6-b78d-76393c222427" containerName="manager"
Jul 28 18:19:47.460580 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.459573    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd2632be-e92b-4584-a349-852464063060" containerName="manager"
Jul 28 18:19:47.460580 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.459581    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd950a52-7d1d-40d6-b78d-76393c222427" containerName="manager"
Jul 28 18:19:47.461658 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.461643    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.464632 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.464610    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8zvj5\""
Jul 28 18:19:47.475936 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.475907    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"]
Jul 28 18:19:47.572486 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.572441    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqtxn\" (UniqueName: \"kubernetes.io/projected/2b8047d4-daab-4e14-a51b-9f219c9dad10-kube-api-access-vqtxn\") pod \"kuadrant-operator-controller-manager-86878c769f-5w799\" (UID: \"2b8047d4-daab-4e14-a51b-9f219c9dad10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.572674 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.572616    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b8047d4-daab-4e14-a51b-9f219c9dad10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5w799\" (UID: \"2b8047d4-daab-4e14-a51b-9f219c9dad10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.674056 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.674012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b8047d4-daab-4e14-a51b-9f219c9dad10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5w799\" (UID: \"2b8047d4-daab-4e14-a51b-9f219c9dad10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.674056 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.674065    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqtxn\" (UniqueName: \"kubernetes.io/projected/2b8047d4-daab-4e14-a51b-9f219c9dad10-kube-api-access-vqtxn\") pod \"kuadrant-operator-controller-manager-86878c769f-5w799\" (UID: \"2b8047d4-daab-4e14-a51b-9f219c9dad10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.674428 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.674407    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b8047d4-daab-4e14-a51b-9f219c9dad10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5w799\" (UID: \"2b8047d4-daab-4e14-a51b-9f219c9dad10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.683719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.683688    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqtxn\" (UniqueName: \"kubernetes.io/projected/2b8047d4-daab-4e14-a51b-9f219c9dad10-kube-api-access-vqtxn\") pod \"kuadrant-operator-controller-manager-86878c769f-5w799\" (UID: \"2b8047d4-daab-4e14-a51b-9f219c9dad10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.777219 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.777184    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:47.920629 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:47.920563    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"]
Jul 28 18:19:47.923135 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:19:47.923100    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8047d4_daab_4e14_a51b_9f219c9dad10.slice/crio-3bb6b0d58a9c1ea317edb40a164153115c74e5a8d788ceddf81d6c17925b5b4f WatchSource:0}: Error finding container 3bb6b0d58a9c1ea317edb40a164153115c74e5a8d788ceddf81d6c17925b5b4f: Status 404 returned error can't find the container with id 3bb6b0d58a9c1ea317edb40a164153115c74e5a8d788ceddf81d6c17925b5b4f
Jul 28 18:19:48.735679 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:48.735631    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799" event={"ID":"2b8047d4-daab-4e14-a51b-9f219c9dad10","Type":"ContainerStarted","Data":"251372553a98163731a1e1633556eca0a62ae85481740ea4ec3ad5965a4f15ef"}
Jul 28 18:19:48.736113 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:48.735695    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799" event={"ID":"2b8047d4-daab-4e14-a51b-9f219c9dad10","Type":"ContainerStarted","Data":"3bb6b0d58a9c1ea317edb40a164153115c74e5a8d788ceddf81d6c17925b5b4f"}
Jul 28 18:19:48.736113 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:48.735725    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:19:48.756973 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:48.756920    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799" podStartSLOduration=1.756899556 podStartE2EDuration="1.756899556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:19:48.754942277 +0000 UTC m=+1362.900495256" watchObservedRunningTime="2026-07-28 18:19:48.756899556 +0000 UTC m=+1362.902452599"
Jul 28 18:19:59.742930 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:19:59.742894    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w799"
Jul 28 18:20:08.207437 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:20:08.207397    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:20:19.496263 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:20:19.496228    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:20:34.498817 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:20:34.498774    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:20:43.510407 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:20:43.510371    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:20:56.818559 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:20:56.818526    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:21:05.009367 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:21:05.009326    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:21:14.899283 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:21:14.899244    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:21:26.394383 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:21:26.394339    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:21:35.798542 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:21:35.798502    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:21:45.692482 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:21:45.692441    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:21:59.509339 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:21:59.509298    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:22:07.275750 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:07.275721    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:22:07.279935 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:07.279912    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:22:07.332552 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:07.332512    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:22:28.391113 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:28.391030    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:22:36.406296 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:36.406257    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:22:39.704172 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.704138    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"]
Jul 28 18:22:39.707724 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.707701    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.710367 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.710344    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-3c5de1ba-kserve-self-signed-certs\""
Jul 28 18:22:39.725616 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.725579    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"]
Jul 28 18:22:39.794086 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.794053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-dshm\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.794361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.794111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tmp-dir\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.794361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.794197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tls-certs\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.794361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.794252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-home\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.794361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.794325    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-model-cache\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.794602 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.794411    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdx7l\" (UniqueName: \"kubernetes.io/projected/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-kube-api-access-wdx7l\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895254 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895212    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdx7l\" (UniqueName: \"kubernetes.io/projected/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-kube-api-access-wdx7l\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895437 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-dshm\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895437 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895339    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tmp-dir\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895437 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895366    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tls-certs\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895437 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-home\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895612 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-model-cache\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895743 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tmp-dir\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895768    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-home\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.895972 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.895951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-model-cache\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.897722 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.897698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-dshm\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.898048 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.898025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tls-certs\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:39.903961 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:39.903936    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdx7l\" (UniqueName: \"kubernetes.io/projected/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-kube-api-access-wdx7l\") pod \"e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") " pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:40.017033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:40.016996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:40.164817 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:40.164780    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"]
Jul 28 18:22:40.589911 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:40.589872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j" event={"ID":"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde","Type":"ContainerStarted","Data":"a8ff30951d4b13fc0bd1a51c9b134afca75746449ce12cef940933e9b51f9c09"}
Jul 28 18:22:40.590096 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:40.589920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j" event={"ID":"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde","Type":"ContainerStarted","Data":"26007a4506cb475bea42fb28068e97425506bc1a4cdec79f51650c07a8a811f5"}
Jul 28 18:22:40.607361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:40.607305    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j" podStartSLOduration=1.607288885 podStartE2EDuration="1.607288885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:22:40.606222522 +0000 UTC m=+1534.751775534" watchObservedRunningTime="2026-07-28 18:22:40.607288885 +0000 UTC m=+1534.752841898"
Jul 28 18:22:50.017863 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:50.017818    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:50.018294 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:50.017910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:50.031054 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:50.031023    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:50.653319 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:50.653284    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:22:52.095126 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:52.095090    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:22:59.004383 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.004353    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"]
Jul 28 18:22:59.008317 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.008297    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.010992 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.010964    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-3c5de1ba-kserve-self-signed-certs\""
Jul 28 18:22:59.020509 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.020479    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"]
Jul 28 18:22:59.070518 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.070481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-dshm\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.070518 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.070525    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-model-cache\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.070768 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.070555    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbw7z\" (UniqueName: \"kubernetes.io/projected/33eb4cf9-3956-41c9-a696-5d949074dcf1-kube-api-access-sbw7z\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.070768 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.070622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33eb4cf9-3956-41c9-a696-5d949074dcf1-tls-certs\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.070768 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.070704    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-home\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.070768 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.070739    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-tmp-dir\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.171869 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.171809    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-dshm\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.171891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-model-cache\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.171922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbw7z\" (UniqueName: \"kubernetes.io/projected/33eb4cf9-3956-41c9-a696-5d949074dcf1-kube-api-access-sbw7z\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.171965    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33eb4cf9-3956-41c9-a696-5d949074dcf1-tls-certs\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.172016    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-home\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.172056    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-tmp-dir\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172388 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.172356    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-model-cache\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172508 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.172398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-home\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.172508 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.172484    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-tmp-dir\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.174264 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.174241    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-dshm\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.174483 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.174464    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33eb4cf9-3956-41c9-a696-5d949074dcf1-tls-certs\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.181023 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.180998    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbw7z\" (UniqueName: \"kubernetes.io/projected/33eb4cf9-3956-41c9-a696-5d949074dcf1-kube-api-access-sbw7z\") pod \"e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") " pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.318920 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.318792    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:22:59.467648 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.467621    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"]
Jul 28 18:22:59.469678 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:22:59.469638    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33eb4cf9_3956_41c9_a696_5d949074dcf1.slice/crio-496d3538d0fa2761ff7afd676ee810d3dc58ab8c62f156ac067a29ca44672d10 WatchSource:0}: Error finding container 496d3538d0fa2761ff7afd676ee810d3dc58ab8c62f156ac067a29ca44672d10: Status 404 returned error can't find the container with id 496d3538d0fa2761ff7afd676ee810d3dc58ab8c62f156ac067a29ca44672d10
Jul 28 18:22:59.688116 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.688025    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d" event={"ID":"33eb4cf9-3956-41c9-a696-5d949074dcf1","Type":"ContainerStarted","Data":"23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4"}
Jul 28 18:22:59.688116 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.688075    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d" event={"ID":"33eb4cf9-3956-41c9-a696-5d949074dcf1","Type":"ContainerStarted","Data":"496d3538d0fa2761ff7afd676ee810d3dc58ab8c62f156ac067a29ca44672d10"}
Jul 28 18:22:59.706754 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:22:59.706688    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d" podStartSLOduration=1.706672169 podStartE2EDuration="1.706672169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:22:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:22:59.704616364 +0000 UTC m=+1553.850169352" watchObservedRunningTime="2026-07-28 18:22:59.706672169 +0000 UTC m=+1553.852225184"
Jul 28 18:23:02.626968 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:02.626934    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:23:09.318934 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:09.318900    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:23:09.319415 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:09.318947    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:23:09.331972 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:09.331945    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:23:09.759129 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:09.759101    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:23:16.660151 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:16.660112    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"]
Jul 28 18:23:17.422215 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:17.422182    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"]
Jul 28 18:23:21.792232 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:21.792191    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:23:31.679749 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:31.679704    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j" podUID="deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" containerName="main" containerID="cri-o://a8ff30951d4b13fc0bd1a51c9b134afca75746449ce12cef940933e9b51f9c09" gracePeriod=45
Jul 28 18:23:31.858383 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:31.858348    2575 generic.go:356] "Generic (PLEG): container finished" podID="deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" containerID="a8ff30951d4b13fc0bd1a51c9b134afca75746449ce12cef940933e9b51f9c09" exitCode=0
Jul 28 18:23:31.858538 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:31.858425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j" event={"ID":"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde","Type":"ContainerDied","Data":"a8ff30951d4b13fc0bd1a51c9b134afca75746449ce12cef940933e9b51f9c09"}
Jul 28 18:23:31.942165 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:31.942082    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:23:32.000451 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000415    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdx7l\" (UniqueName: \"kubernetes.io/projected/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-kube-api-access-wdx7l\") pod \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") "
Jul 28 18:23:32.000630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000520    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-home\") pod \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") "
Jul 28 18:23:32.000630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000552    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-model-cache\") pod \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") "
Jul 28 18:23:32.000630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000569    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-dshm\") pod \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") "
Jul 28 18:23:32.000630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000593    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tls-certs\") pod \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") "
Jul 28 18:23:32.000885 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000669    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tmp-dir\") pod \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\" (UID: \"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde\") "
Jul 28 18:23:32.000885 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000841    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-home" (OuterVolumeSpecName: "home") pod "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" (UID: "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.000994 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.000889    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-model-cache" (OuterVolumeSpecName: "model-cache") pod "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" (UID: "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.001047 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.001035    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.001108 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.001052    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.001108 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.001062    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" (UID: "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.003382 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.003346    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-dshm" (OuterVolumeSpecName: "dshm") pod "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" (UID: "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.003382 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.003375    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" (UID: "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:23:32.003529 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.003412    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-kube-api-access-wdx7l" (OuterVolumeSpecName: "kube-api-access-wdx7l") pod "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" (UID: "deecdb1e-7bab-43f1-a0e6-3e5419c9ccde"). InnerVolumeSpecName "kube-api-access-wdx7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:23:32.102424 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.102385    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.102424 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.102416    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.102424 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.102426    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.102424 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.102435    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdx7l\" (UniqueName: \"kubernetes.io/projected/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde-kube-api-access-wdx7l\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.444518 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.444442    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d" podUID="33eb4cf9-3956-41c9-a696-5d949074dcf1" containerName="main" containerID="cri-o://23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4" gracePeriod=45
Jul 28 18:23:32.690275 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.690244    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:23:32.707727 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.707692    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33eb4cf9-3956-41c9-a696-5d949074dcf1-tls-certs\") pod \"33eb4cf9-3956-41c9-a696-5d949074dcf1\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") "
Jul 28 18:23:32.707950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.707750    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-tmp-dir\") pod \"33eb4cf9-3956-41c9-a696-5d949074dcf1\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") "
Jul 28 18:23:32.707950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.707775    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbw7z\" (UniqueName: \"kubernetes.io/projected/33eb4cf9-3956-41c9-a696-5d949074dcf1-kube-api-access-sbw7z\") pod \"33eb4cf9-3956-41c9-a696-5d949074dcf1\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") "
Jul 28 18:23:32.707950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.707843    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-dshm\") pod \"33eb4cf9-3956-41c9-a696-5d949074dcf1\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") "
Jul 28 18:23:32.708114 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.707967    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-model-cache\") pod \"33eb4cf9-3956-41c9-a696-5d949074dcf1\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") "
Jul 28 18:23:32.708114 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.707994    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-home\") pod \"33eb4cf9-3956-41c9-a696-5d949074dcf1\" (UID: \"33eb4cf9-3956-41c9-a696-5d949074dcf1\") "
Jul 28 18:23:32.708114 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.708068    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33eb4cf9-3956-41c9-a696-5d949074dcf1" (UID: "33eb4cf9-3956-41c9-a696-5d949074dcf1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.708284 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.708254    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-home" (OuterVolumeSpecName: "home") pod "33eb4cf9-3956-41c9-a696-5d949074dcf1" (UID: "33eb4cf9-3956-41c9-a696-5d949074dcf1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.708284 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.708238    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-model-cache" (OuterVolumeSpecName: "model-cache") pod "33eb4cf9-3956-41c9-a696-5d949074dcf1" (UID: "33eb4cf9-3956-41c9-a696-5d949074dcf1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.708409 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.708385    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.710338 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.710305    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33eb4cf9-3956-41c9-a696-5d949074dcf1-kube-api-access-sbw7z" (OuterVolumeSpecName: "kube-api-access-sbw7z") pod "33eb4cf9-3956-41c9-a696-5d949074dcf1" (UID: "33eb4cf9-3956-41c9-a696-5d949074dcf1"). InnerVolumeSpecName "kube-api-access-sbw7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:23:32.710338 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.710332    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33eb4cf9-3956-41c9-a696-5d949074dcf1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33eb4cf9-3956-41c9-a696-5d949074dcf1" (UID: "33eb4cf9-3956-41c9-a696-5d949074dcf1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:23:32.710549 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.710315    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-dshm" (OuterVolumeSpecName: "dshm") pod "33eb4cf9-3956-41c9-a696-5d949074dcf1" (UID: "33eb4cf9-3956-41c9-a696-5d949074dcf1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:23:32.809265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.809215    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.809265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.809256    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.809265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.809269    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33eb4cf9-3956-41c9-a696-5d949074dcf1-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.809515 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.809280    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbw7z\" (UniqueName: \"kubernetes.io/projected/33eb4cf9-3956-41c9-a696-5d949074dcf1-kube-api-access-sbw7z\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.809515 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.809293    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33eb4cf9-3956-41c9-a696-5d949074dcf1-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:23:32.864193 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.864163    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"
Jul 28 18:23:32.864390 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.864200    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j" event={"ID":"deecdb1e-7bab-43f1-a0e6-3e5419c9ccde","Type":"ContainerDied","Data":"26007a4506cb475bea42fb28068e97425506bc1a4cdec79f51650c07a8a811f5"}
Jul 28 18:23:32.864390 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.864247    2575 scope.go:117] "RemoveContainer" containerID="a8ff30951d4b13fc0bd1a51c9b134afca75746449ce12cef940933e9b51f9c09"
Jul 28 18:23:32.865694 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.865672    2575 generic.go:356] "Generic (PLEG): container finished" podID="33eb4cf9-3956-41c9-a696-5d949074dcf1" containerID="23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4" exitCode=0
Jul 28 18:23:32.865781 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.865721    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d" event={"ID":"33eb4cf9-3956-41c9-a696-5d949074dcf1","Type":"ContainerDied","Data":"23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4"}
Jul 28 18:23:32.865781 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.865739    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"
Jul 28 18:23:32.865781 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.865750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d" event={"ID":"33eb4cf9-3956-41c9-a696-5d949074dcf1","Type":"ContainerDied","Data":"496d3538d0fa2761ff7afd676ee810d3dc58ab8c62f156ac067a29ca44672d10"}
Jul 28 18:23:32.880739 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.880717    2575 scope.go:117] "RemoveContainer" containerID="23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4"
Jul 28 18:23:32.887404 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.887377    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"]
Jul 28 18:23:32.890144 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.890113    2575 scope.go:117] "RemoveContainer" containerID="23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4"
Jul 28 18:23:32.890506 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:23:32.890485    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4\": container with ID starting with 23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4 not found: ID does not exist" containerID="23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4"
Jul 28 18:23:32.890574 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.890516    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4"} err="failed to get container status \"23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4\": rpc error: code = NotFound desc = could not find container \"23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4\": container with ID starting with 23ad92ebc6b4d0d7719d8a0c383a1e59b5fc5eacc52f907acc9032dec6cdb7c4 not found: ID does not exist"
Jul 28 18:23:32.891117 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.891100    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-3c5de1ba-kserve-6877b4867f-8bh5j"]
Jul 28 18:23:32.904915 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.904881    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"]
Jul 28 18:23:32.908432 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:32.908406    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-3c5de1ba-kserve-68f6ff8575-fgh4d"]
Jul 28 18:23:33.723449 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:33.723411    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:23:34.400219 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:34.400188    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33eb4cf9-3956-41c9-a696-5d949074dcf1" path="/var/lib/kubelet/pods/33eb4cf9-3956-41c9-a696-5d949074dcf1/volumes"
Jul 28 18:23:34.400594 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:34.400580    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" path="/var/lib/kubelet/pods/deecdb1e-7bab-43f1-a0e6-3e5419c9ccde/volumes"
Jul 28 18:23:42.897873 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:42.897814    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:23:55.597300 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:23:55.597262    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:24:10.392722 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:24:10.392682    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:24:19.912614 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:24:19.912575    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:24:34.094692 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:24:34.094651    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:24:43.594319 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:24:43.594276    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:24:57.295796 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:24:57.295754    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:25:06.591612 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:25:06.591578    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:25:16.298213 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:25:16.298159    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:25:26.591351 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:25:26.591315    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:25:38.704202 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:25:38.704122    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:25:48.196171 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:25:48.196132    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:25:59.994518 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:25:59.994476    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:26:12.703490 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:26:12.703454    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:26:21.796008 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:26:21.795968    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:26:32.699433 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:26:32.699395    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:26:42.005409 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:26:42.005369    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:27:04.276704 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:27:04.276665    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:27:07.302387 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:27:07.302359    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:27:07.315530 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:27:07.315499    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:27:19.003098 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:27:19.003057    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:27:49.926260 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:27:49.926223    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:28:00.204538 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:28:00.204499    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:28:24.393597 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:28:24.393564    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:28:34.908515 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:28:34.908431    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:28:59.288491 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:28:59.288454    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:29:09.900304 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:29:09.900265    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:29:34.698895 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:29:34.698832    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:29:44.590963 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:29:44.590925    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:30:59.295802 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:30:59.295764    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:31:08.922899 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:31:08.922848    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:31:19.305177 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:31:19.305144    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:31:30.205045 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:31:30.204958    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:32:05.810479 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:32:05.810439    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:32:07.338579 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:32:07.338550    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:32:07.346461 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:32:07.346432    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:32:39.398957 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:32:39.398914    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:33:06.295323 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.295253    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"]
Jul 28 18:33:06.295747 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.295633    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33eb4cf9-3956-41c9-a696-5d949074dcf1" containerName="main"
Jul 28 18:33:06.295747 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.295645    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="33eb4cf9-3956-41c9-a696-5d949074dcf1" containerName="main"
Jul 28 18:33:06.295747 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.295669    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" containerName="main"
Jul 28 18:33:06.295747 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.295675    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" containerName="main"
Jul 28 18:33:06.295747 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.295736    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="deecdb1e-7bab-43f1-a0e6-3e5419c9ccde" containerName="main"
Jul 28 18:33:06.295747 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.295745    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="33eb4cf9-3956-41c9-a696-5d949074dcf1" containerName="main"
Jul 28 18:33:06.298525 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.298504    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.301194 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.301171    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-06daccab\"/\"kube-root-ca.crt\""
Jul 28 18:33:06.302236 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.302213    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-06daccab\"/\"e2e-lifecycle-model-06daccab-kserve-self-signed-certs\""
Jul 28 18:33:06.302236 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.302226    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-06daccab\"/\"openshift-service-ca.crt\""
Jul 28 18:33:06.302380 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.302219    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-06daccab\"/\"default-dockercfg-kz9rq\""
Jul 28 18:33:06.308222 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.308198    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"]
Jul 28 18:33:06.386101 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.386053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-home\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.386295 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.386117    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8z22\" (UniqueName: \"kubernetes.io/projected/880d024a-e372-48f3-929e-c2ee716586ab-kube-api-access-c8z22\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.386295 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.386175    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-model-cache\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.386295 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.386256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-tmp-dir\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.386428 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.386332    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/880d024a-e372-48f3-929e-c2ee716586ab-tls-certs\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.386428 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.386368    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-dshm\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.486805 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.486764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-home\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.486805 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.486808    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8z22\" (UniqueName: \"kubernetes.io/projected/880d024a-e372-48f3-929e-c2ee716586ab-kube-api-access-c8z22\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.487070 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.486835    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-model-cache\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.487070 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.486883    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-tmp-dir\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.487070 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.486916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/880d024a-e372-48f3-929e-c2ee716586ab-tls-certs\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.487070 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.486952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-dshm\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.487596 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.487197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-home\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.487596 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.487238    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-model-cache\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.487596 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.487469    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-tmp-dir\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.489284 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.489264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-dshm\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.489756 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.489735    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/880d024a-e372-48f3-929e-c2ee716586ab-tls-certs\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.495775 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.495742    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8z22\" (UniqueName: \"kubernetes.io/projected/880d024a-e372-48f3-929e-c2ee716586ab-kube-api-access-c8z22\") pod \"e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") " pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.609106 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.608997    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:06.758483 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:06.758446    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"]
Jul 28 18:33:06.762698 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:33:06.762666    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod880d024a_e372_48f3_929e_c2ee716586ab.slice/crio-71006d23b6878389c4127bcf7606befae8f18085f239f91601819d982d262ac9 WatchSource:0}: Error finding container 71006d23b6878389c4127bcf7606befae8f18085f239f91601819d982d262ac9: Status 404 returned error can't find the container with id 71006d23b6878389c4127bcf7606befae8f18085f239f91601819d982d262ac9
Jul 28 18:33:07.719083 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:07.719044    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n" event={"ID":"880d024a-e372-48f3-929e-c2ee716586ab","Type":"ContainerStarted","Data":"83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912"}
Jul 28 18:33:07.719493 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:07.719095    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n" event={"ID":"880d024a-e372-48f3-929e-c2ee716586ab","Type":"ContainerStarted","Data":"71006d23b6878389c4127bcf7606befae8f18085f239f91601819d982d262ac9"}
Jul 28 18:33:07.736810 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:07.736754    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n" podStartSLOduration=1.736736102 podStartE2EDuration="1.736736102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:33:07.734973023 +0000 UTC m=+2161.880525999" watchObservedRunningTime="2026-07-28 18:33:07.736736102 +0000 UTC m=+2161.882289077"
Jul 28 18:33:09.791011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:09.790972    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:33:16.610086 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:16.610020    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:16.610086 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:16.610097    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:16.622702 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:16.622673    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:16.777034 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:16.776999    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:33:29.908828 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:29.908789    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:33:51.394518 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:33:51.394470    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:34:01.525780 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:01.525737    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"]
Jul 28 18:34:02.597125 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:02.597091    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:34:16.545735 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.545687    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n" podUID="880d024a-e372-48f3-929e-c2ee716586ab" containerName="main" containerID="cri-o://83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912" gracePeriod=45
Jul 28 18:34:16.805681 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.805608    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:34:16.940651 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.940614    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-model-cache\") pod \"880d024a-e372-48f3-929e-c2ee716586ab\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") "
Jul 28 18:34:16.940845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.940662    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8z22\" (UniqueName: \"kubernetes.io/projected/880d024a-e372-48f3-929e-c2ee716586ab-kube-api-access-c8z22\") pod \"880d024a-e372-48f3-929e-c2ee716586ab\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") "
Jul 28 18:34:16.940845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.940694    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-tmp-dir\") pod \"880d024a-e372-48f3-929e-c2ee716586ab\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") "
Jul 28 18:34:16.940845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.940719    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-dshm\") pod \"880d024a-e372-48f3-929e-c2ee716586ab\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") "
Jul 28 18:34:16.940845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.940781    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/880d024a-e372-48f3-929e-c2ee716586ab-tls-certs\") pod \"880d024a-e372-48f3-929e-c2ee716586ab\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") "
Jul 28 18:34:16.940845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.940819    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-home\") pod \"880d024a-e372-48f3-929e-c2ee716586ab\" (UID: \"880d024a-e372-48f3-929e-c2ee716586ab\") "
Jul 28 18:34:16.941149 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.940990    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-model-cache" (OuterVolumeSpecName: "model-cache") pod "880d024a-e372-48f3-929e-c2ee716586ab" (UID: "880d024a-e372-48f3-929e-c2ee716586ab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:16.941149 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.941059    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "880d024a-e372-48f3-929e-c2ee716586ab" (UID: "880d024a-e372-48f3-929e-c2ee716586ab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:16.941263 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.941203    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:34:16.941263 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.941205    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-home" (OuterVolumeSpecName: "home") pod "880d024a-e372-48f3-929e-c2ee716586ab" (UID: "880d024a-e372-48f3-929e-c2ee716586ab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:16.941263 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.941223    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:34:16.943196 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.943164    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/880d024a-e372-48f3-929e-c2ee716586ab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "880d024a-e372-48f3-929e-c2ee716586ab" (UID: "880d024a-e372-48f3-929e-c2ee716586ab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:34:16.943196 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.943190    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-dshm" (OuterVolumeSpecName: "dshm") pod "880d024a-e372-48f3-929e-c2ee716586ab" (UID: "880d024a-e372-48f3-929e-c2ee716586ab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:34:16.943401 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:16.943229    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/880d024a-e372-48f3-929e-c2ee716586ab-kube-api-access-c8z22" (OuterVolumeSpecName: "kube-api-access-c8z22") pod "880d024a-e372-48f3-929e-c2ee716586ab" (UID: "880d024a-e372-48f3-929e-c2ee716586ab"). InnerVolumeSpecName "kube-api-access-c8z22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:34:17.042738 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.042697    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8z22\" (UniqueName: \"kubernetes.io/projected/880d024a-e372-48f3-929e-c2ee716586ab-kube-api-access-c8z22\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:34:17.042738 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.042728    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:34:17.042738 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.042739    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/880d024a-e372-48f3-929e-c2ee716586ab-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:34:17.042738 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.042748    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/880d024a-e372-48f3-929e-c2ee716586ab-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:34:17.062456 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.062365    2575 generic.go:356] "Generic (PLEG): container finished" podID="880d024a-e372-48f3-929e-c2ee716586ab" containerID="83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912" exitCode=0
Jul 28 18:34:17.062456 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.062409    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"
Jul 28 18:34:17.062633 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.062457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n" event={"ID":"880d024a-e372-48f3-929e-c2ee716586ab","Type":"ContainerDied","Data":"83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912"}
Jul 28 18:34:17.062633 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.062505    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n" event={"ID":"880d024a-e372-48f3-929e-c2ee716586ab","Type":"ContainerDied","Data":"71006d23b6878389c4127bcf7606befae8f18085f239f91601819d982d262ac9"}
Jul 28 18:34:17.062633 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.062524    2575 scope.go:117] "RemoveContainer" containerID="83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912"
Jul 28 18:34:17.071815 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.071798    2575 scope.go:117] "RemoveContainer" containerID="83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912"
Jul 28 18:34:17.072172 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:34:17.072152    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912\": container with ID starting with 83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912 not found: ID does not exist" containerID="83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912"
Jul 28 18:34:17.072231 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.072182    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912"} err="failed to get container status \"83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912\": rpc error: code = NotFound desc = could not find container \"83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912\": container with ID starting with 83694c88e176210673a1678210e670d09da2a3b7a5b06ea3fb515347d1da3912 not found: ID does not exist"
Jul 28 18:34:17.084864 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.084819    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"]
Jul 28 18:34:17.092206 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:17.092167    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n"]
Jul 28 18:34:18.400649 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:18.400605    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="880d024a-e372-48f3-929e-c2ee716586ab" path="/var/lib/kubelet/pods/880d024a-e372-48f3-929e-c2ee716586ab/volumes"
Jul 28 18:34:21.766516 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:21.766459    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-06daccab/e2e-lifecycle-model-06daccab-kserve-b67d68cf-7tj7n" podUID="880d024a-e372-48f3-929e-c2ee716586ab" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Jul 28 18:34:48.899087 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:34:48.899044    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:35:01.900904 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:35:01.900842    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:35:50.598286 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:35:50.598229    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:36:26.102224 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:26.102142    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:36:48.793229 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.793190    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"]
Jul 28 18:36:48.793630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.793558    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880d024a-e372-48f3-929e-c2ee716586ab" containerName="main"
Jul 28 18:36:48.793630 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.793569    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="880d024a-e372-48f3-929e-c2ee716586ab" containerName="main"
Jul 28 18:36:48.793706 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.793643    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="880d024a-e372-48f3-929e-c2ee716586ab" containerName="main"
Jul 28 18:36:48.796507 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.796489    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.799206 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.799175    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b228ad\"/\"openshift-service-ca.crt\""
Jul 28 18:36:48.799372 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.799242    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b228ad\"/\"default-dockercfg-wd2n9\""
Jul 28 18:36:48.800215 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.800193    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b228ad\"/\"test-model-b228ad-kserve-self-signed-certs\""
Jul 28 18:36:48.800328 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.800248    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b228ad\"/\"kube-root-ca.crt\""
Jul 28 18:36:48.806904 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.806881    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"]
Jul 28 18:36:48.849069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.849033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-dshm\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.849069 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.849082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/877a5ecc-1a59-49c9-b78e-968864e2496c-tls-certs\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.849405 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.849109    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s75mj\" (UniqueName: \"kubernetes.io/projected/877a5ecc-1a59-49c9-b78e-968864e2496c-kube-api-access-s75mj\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.849405 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.849135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-model-cache\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.849405 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.849207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-tmp-dir\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.849405 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.849260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-home\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.950521 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.950483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-home\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.950725 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.950555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-dshm\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.950725 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.950671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/877a5ecc-1a59-49c9-b78e-968864e2496c-tls-certs\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.950725 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.950703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s75mj\" (UniqueName: \"kubernetes.io/projected/877a5ecc-1a59-49c9-b78e-968864e2496c-kube-api-access-s75mj\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.950927 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.950732    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-model-cache\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.950927 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.950793    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-tmp-dir\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.951072 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.951049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-home\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.951161 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.951142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-model-cache\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.951219 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.951203    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-tmp-dir\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.952893 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.952838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-dshm\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.953148 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.953130    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/877a5ecc-1a59-49c9-b78e-968864e2496c-tls-certs\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:48.959431 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:48.959403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s75mj\" (UniqueName: \"kubernetes.io/projected/877a5ecc-1a59-49c9-b78e-968864e2496c-kube-api-access-s75mj\") pod \"test-model-b228ad-kserve-55695b8c55-f6vfv\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") " pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:49.106770 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:49.106664    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:49.256428 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:49.256403    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"]
Jul 28 18:36:49.258745 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:36:49.258712    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod877a5ecc_1a59_49c9_b78e_968864e2496c.slice/crio-6d0a34652fb8afc54959bc04709833c9f7af476d74efc4b26cdea1612c8fb3c1 WatchSource:0}: Error finding container 6d0a34652fb8afc54959bc04709833c9f7af476d74efc4b26cdea1612c8fb3c1: Status 404 returned error can't find the container with id 6d0a34652fb8afc54959bc04709833c9f7af476d74efc4b26cdea1612c8fb3c1
Jul 28 18:36:49.836953 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:49.836910    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv" event={"ID":"877a5ecc-1a59-49c9-b78e-968864e2496c","Type":"ContainerStarted","Data":"5deec58b48cf22264e28a34c1b26bdec2b9e1b4b68e239fd863a6384915a4bf6"}
Jul 28 18:36:49.837364 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:49.836964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv" event={"ID":"877a5ecc-1a59-49c9-b78e-968864e2496c","Type":"ContainerStarted","Data":"6d0a34652fb8afc54959bc04709833c9f7af476d74efc4b26cdea1612c8fb3c1"}
Jul 28 18:36:49.856576 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:49.856516    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv" podStartSLOduration=1.856495952 podStartE2EDuration="1.856495952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:36:49.855160408 +0000 UTC m=+2384.000713385" watchObservedRunningTime="2026-07-28 18:36:49.856495952 +0000 UTC m=+2384.002048925"
Jul 28 18:36:53.588405 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:53.588366    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:36:59.107491 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:59.107453    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:59.107491 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:59.107505    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:59.120223 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:59.120191    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:36:59.898920 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:36:59.898892    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:37:07.366958 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:07.366931    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:37:07.374505 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:07.374480    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:37:23.699637 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.699597    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"]
Jul 28 18:37:23.702870 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.702833    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.705553 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.705530    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-210335\"/\"test-model-210335-kserve-self-signed-certs\""
Jul 28 18:37:23.706604 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.706583    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-210335\"/\"default-dockercfg-bfh6f\""
Jul 28 18:37:23.706700 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.706620    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-210335\"/\"kube-root-ca.crt\""
Jul 28 18:37:23.706700 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.706622    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-210335\"/\"openshift-service-ca.crt\""
Jul 28 18:37:23.714210 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.714184    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"]
Jul 28 18:37:23.776222 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.776188    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c616fe-457d-4029-a440-5374a3a05a6b-tls-certs\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.776392 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.776232    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-tmp-dir\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.776392 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.776285    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-model-cache\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.776392 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.776339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-dshm\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.776392 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.776369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smgzx\" (UniqueName: \"kubernetes.io/projected/06c616fe-457d-4029-a440-5374a3a05a6b-kube-api-access-smgzx\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.776533 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.776446    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-home\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.877481 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.877440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-dshm\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.877481 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.877486    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smgzx\" (UniqueName: \"kubernetes.io/projected/06c616fe-457d-4029-a440-5374a3a05a6b-kube-api-access-smgzx\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.877758 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.877525    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-home\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.877758 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.877551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c616fe-457d-4029-a440-5374a3a05a6b-tls-certs\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.877758 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.877575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-tmp-dir\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.877758 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.877599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-model-cache\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.878027 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.878006    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-home\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.878070 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.878049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-tmp-dir\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.878115 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.878076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-model-cache\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.879997 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.879974    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-dshm\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.880173 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.880155    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c616fe-457d-4029-a440-5374a3a05a6b-tls-certs\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:23.886276 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:23.886245    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smgzx\" (UniqueName: \"kubernetes.io/projected/06c616fe-457d-4029-a440-5374a3a05a6b-kube-api-access-smgzx\") pod \"test-model-210335-kserve-7996b79fd5-4zzkw\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") " pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:24.012958 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:24.012914    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:24.181940 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:24.181912    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"]
Jul 28 18:37:24.183805 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:37:24.183767    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06c616fe_457d_4029_a440_5374a3a05a6b.slice/crio-ad83d91224864915eb97256d9b4b7e58cda73be05c649a7900570637fa358db8 WatchSource:0}: Error finding container ad83d91224864915eb97256d9b4b7e58cda73be05c649a7900570637fa358db8: Status 404 returned error can't find the container with id ad83d91224864915eb97256d9b4b7e58cda73be05c649a7900570637fa358db8
Jul 28 18:37:25.008517 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:25.008481    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw" event={"ID":"06c616fe-457d-4029-a440-5374a3a05a6b","Type":"ContainerStarted","Data":"b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28"}
Jul 28 18:37:25.008517 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:25.008523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw" event={"ID":"06c616fe-457d-4029-a440-5374a3a05a6b","Type":"ContainerStarted","Data":"ad83d91224864915eb97256d9b4b7e58cda73be05c649a7900570637fa358db8"}
Jul 28 18:37:25.026669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:25.026611    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw" podStartSLOduration=2.026592304 podStartE2EDuration="2.026592304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:37:25.024701968 +0000 UTC m=+2419.170254966" watchObservedRunningTime="2026-07-28 18:37:25.026592304 +0000 UTC m=+2419.172145278"
Jul 28 18:37:27.788205 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:27.788116    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:37:34.013369 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:34.013329    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:34.013369 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:34.013377    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:34.026126 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:34.026094    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:34.064724 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:34.064692    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:37:41.699280 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:37:41.699240    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:38:04.007133 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:04.007094    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"]
Jul 28 18:38:04.940282 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:04.940243    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"]
Jul 28 18:38:09.189962 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:09.189928    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:38:19.027005 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.026939    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv" podUID="877a5ecc-1a59-49c9-b78e-968864e2496c" containerName="main" containerID="cri-o://5deec58b48cf22264e28a34c1b26bdec2b9e1b4b68e239fd863a6384915a4bf6" gracePeriod=45
Jul 28 18:38:19.277271 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.277173    2575 generic.go:356] "Generic (PLEG): container finished" podID="877a5ecc-1a59-49c9-b78e-968864e2496c" containerID="5deec58b48cf22264e28a34c1b26bdec2b9e1b4b68e239fd863a6384915a4bf6" exitCode=0
Jul 28 18:38:19.277271 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.277254    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv" event={"ID":"877a5ecc-1a59-49c9-b78e-968864e2496c","Type":"ContainerDied","Data":"5deec58b48cf22264e28a34c1b26bdec2b9e1b4b68e239fd863a6384915a4bf6"}
Jul 28 18:38:19.277510 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.277300    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv" event={"ID":"877a5ecc-1a59-49c9-b78e-968864e2496c","Type":"ContainerDied","Data":"6d0a34652fb8afc54959bc04709833c9f7af476d74efc4b26cdea1612c8fb3c1"}
Jul 28 18:38:19.277510 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.277311    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d0a34652fb8afc54959bc04709833c9f7af476d74efc4b26cdea1612c8fb3c1"
Jul 28 18:38:19.286137 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.286107    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:38:19.389168 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389137    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s75mj\" (UniqueName: \"kubernetes.io/projected/877a5ecc-1a59-49c9-b78e-968864e2496c-kube-api-access-s75mj\") pod \"877a5ecc-1a59-49c9-b78e-968864e2496c\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") "
Jul 28 18:38:19.389381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389199    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/877a5ecc-1a59-49c9-b78e-968864e2496c-tls-certs\") pod \"877a5ecc-1a59-49c9-b78e-968864e2496c\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") "
Jul 28 18:38:19.389381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389225    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-dshm\") pod \"877a5ecc-1a59-49c9-b78e-968864e2496c\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") "
Jul 28 18:38:19.389381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389262    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-model-cache\") pod \"877a5ecc-1a59-49c9-b78e-968864e2496c\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") "
Jul 28 18:38:19.389381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389284    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-home\") pod \"877a5ecc-1a59-49c9-b78e-968864e2496c\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") "
Jul 28 18:38:19.389381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389327    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-tmp-dir\") pod \"877a5ecc-1a59-49c9-b78e-968864e2496c\" (UID: \"877a5ecc-1a59-49c9-b78e-968864e2496c\") "
Jul 28 18:38:19.389646 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389520    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-model-cache" (OuterVolumeSpecName: "model-cache") pod "877a5ecc-1a59-49c9-b78e-968864e2496c" (UID: "877a5ecc-1a59-49c9-b78e-968864e2496c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:19.389646 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389564    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-home" (OuterVolumeSpecName: "home") pod "877a5ecc-1a59-49c9-b78e-968864e2496c" (UID: "877a5ecc-1a59-49c9-b78e-968864e2496c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:19.389753 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389636    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "877a5ecc-1a59-49c9-b78e-968864e2496c" (UID: "877a5ecc-1a59-49c9-b78e-968864e2496c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:19.389753 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389666    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:19.389753 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.389683    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:19.391596 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.391574    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-dshm" (OuterVolumeSpecName: "dshm") pod "877a5ecc-1a59-49c9-b78e-968864e2496c" (UID: "877a5ecc-1a59-49c9-b78e-968864e2496c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:19.391895 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.391871    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877a5ecc-1a59-49c9-b78e-968864e2496c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "877a5ecc-1a59-49c9-b78e-968864e2496c" (UID: "877a5ecc-1a59-49c9-b78e-968864e2496c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:38:19.391895 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.391877    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877a5ecc-1a59-49c9-b78e-968864e2496c-kube-api-access-s75mj" (OuterVolumeSpecName: "kube-api-access-s75mj") pod "877a5ecc-1a59-49c9-b78e-968864e2496c" (UID: "877a5ecc-1a59-49c9-b78e-968864e2496c"). InnerVolumeSpecName "kube-api-access-s75mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:19.490808 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.490771    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s75mj\" (UniqueName: \"kubernetes.io/projected/877a5ecc-1a59-49c9-b78e-968864e2496c-kube-api-access-s75mj\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:19.490808 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.490801    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/877a5ecc-1a59-49c9-b78e-968864e2496c-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:19.490808 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.490814    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:19.491100 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.490825    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/877a5ecc-1a59-49c9-b78e-968864e2496c-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:19.960386 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:19.960324    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw" podUID="06c616fe-457d-4029-a440-5374a3a05a6b" containerName="main" containerID="cri-o://b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28" gracePeriod=45
Jul 28 18:38:20.225356 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.225275    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:38:20.283906 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.283873    2575 generic.go:356] "Generic (PLEG): container finished" podID="06c616fe-457d-4029-a440-5374a3a05a6b" containerID="b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28" exitCode=0
Jul 28 18:38:20.284102 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.283918    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"
Jul 28 18:38:20.284102 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.283956    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw" event={"ID":"06c616fe-457d-4029-a440-5374a3a05a6b","Type":"ContainerDied","Data":"b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28"}
Jul 28 18:38:20.284102 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.283995    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw" event={"ID":"06c616fe-457d-4029-a440-5374a3a05a6b","Type":"ContainerDied","Data":"ad83d91224864915eb97256d9b4b7e58cda73be05c649a7900570637fa358db8"}
Jul 28 18:38:20.284102 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.284013    2575 scope.go:117] "RemoveContainer" containerID="b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28"
Jul 28 18:38:20.284102 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.284055    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"
Jul 28 18:38:20.293403 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.293385    2575 scope.go:117] "RemoveContainer" containerID="b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28"
Jul 28 18:38:20.293717 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:38:20.293694    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28\": container with ID starting with b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28 not found: ID does not exist" containerID="b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28"
Jul 28 18:38:20.293762 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.293727    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28"} err="failed to get container status \"b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28\": rpc error: code = NotFound desc = could not find container \"b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28\": container with ID starting with b3ce67195de0786a3a772f092046f01a21a487980f536fec8058b1cdcc480c28 not found: ID does not exist"
Jul 28 18:38:20.299591 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.299565    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-tmp-dir\") pod \"06c616fe-457d-4029-a440-5374a3a05a6b\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") "
Jul 28 18:38:20.299709 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.299602    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c616fe-457d-4029-a440-5374a3a05a6b-tls-certs\") pod \"06c616fe-457d-4029-a440-5374a3a05a6b\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") "
Jul 28 18:38:20.299709 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.299620    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-dshm\") pod \"06c616fe-457d-4029-a440-5374a3a05a6b\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") "
Jul 28 18:38:20.299781 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.299742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-home\") pod \"06c616fe-457d-4029-a440-5374a3a05a6b\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") "
Jul 28 18:38:20.299816 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.299786    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smgzx\" (UniqueName: \"kubernetes.io/projected/06c616fe-457d-4029-a440-5374a3a05a6b-kube-api-access-smgzx\") pod \"06c616fe-457d-4029-a440-5374a3a05a6b\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") "
Jul 28 18:38:20.299892 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.299817    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-model-cache\") pod \"06c616fe-457d-4029-a440-5374a3a05a6b\" (UID: \"06c616fe-457d-4029-a440-5374a3a05a6b\") "
Jul 28 18:38:20.299948 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.299897    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06c616fe-457d-4029-a440-5374a3a05a6b" (UID: "06c616fe-457d-4029-a440-5374a3a05a6b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:20.300076 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.300049    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-model-cache" (OuterVolumeSpecName: "model-cache") pod "06c616fe-457d-4029-a440-5374a3a05a6b" (UID: "06c616fe-457d-4029-a440-5374a3a05a6b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:20.300076 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.300062    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-home" (OuterVolumeSpecName: "home") pod "06c616fe-457d-4029-a440-5374a3a05a6b" (UID: "06c616fe-457d-4029-a440-5374a3a05a6b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:20.300197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.300133    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:20.300197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.300148    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:20.300197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.300158    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:20.302032 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.302008    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c616fe-457d-4029-a440-5374a3a05a6b-kube-api-access-smgzx" (OuterVolumeSpecName: "kube-api-access-smgzx") pod "06c616fe-457d-4029-a440-5374a3a05a6b" (UID: "06c616fe-457d-4029-a440-5374a3a05a6b"). InnerVolumeSpecName "kube-api-access-smgzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:20.302305 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.302285    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-dshm" (OuterVolumeSpecName: "dshm") pod "06c616fe-457d-4029-a440-5374a3a05a6b" (UID: "06c616fe-457d-4029-a440-5374a3a05a6b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:38:20.302358 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.302296    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06c616fe-457d-4029-a440-5374a3a05a6b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06c616fe-457d-4029-a440-5374a3a05a6b" (UID: "06c616fe-457d-4029-a440-5374a3a05a6b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:38:20.305689 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.305648    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"]
Jul 28 18:38:20.310582 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.310556    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b228ad/test-model-b228ad-kserve-55695b8c55-f6vfv"]
Jul 28 18:38:20.401264 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.401228    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smgzx\" (UniqueName: \"kubernetes.io/projected/06c616fe-457d-4029-a440-5374a3a05a6b-kube-api-access-smgzx\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:20.401264 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.401263    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c616fe-457d-4029-a440-5374a3a05a6b-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:20.401527 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.401277    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06c616fe-457d-4029-a440-5374a3a05a6b-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:38:20.403693 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.403661    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877a5ecc-1a59-49c9-b78e-968864e2496c" path="/var/lib/kubelet/pods/877a5ecc-1a59-49c9-b78e-968864e2496c/volumes"
Jul 28 18:38:20.600065 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.600025    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"]
Jul 28 18:38:20.604258 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:20.604227    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-210335/test-model-210335-kserve-7996b79fd5-4zzkw"]
Jul 28 18:38:22.401991 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:22.401959    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06c616fe-457d-4029-a440-5374a3a05a6b" path="/var/lib/kubelet/pods/06c616fe-457d-4029-a440-5374a3a05a6b/volumes"
Jul 28 18:38:24.794204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:24.794163    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:38:36.389231 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:36.389190    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:38:51.003591 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:38:51.003550    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:39:19.196606 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:39:19.196574    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:40:08.691296 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:40:08.691253    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:40:29.900498 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:40:29.900417    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:40:41.096710 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:40:41.096672    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:41:12.192503 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:12.192463    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:41:39.596873 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.596824    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"]
Jul 28 18:41:39.597265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.597220    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877a5ecc-1a59-49c9-b78e-968864e2496c" containerName="main"
Jul 28 18:41:39.597265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.597232    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="877a5ecc-1a59-49c9-b78e-968864e2496c" containerName="main"
Jul 28 18:41:39.597265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.597240    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c616fe-457d-4029-a440-5374a3a05a6b" containerName="main"
Jul 28 18:41:39.597265 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.597246    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c616fe-457d-4029-a440-5374a3a05a6b" containerName="main"
Jul 28 18:41:39.597411 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.597321    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="06c616fe-457d-4029-a440-5374a3a05a6b" containerName="main"
Jul 28 18:41:39.597411 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.597331    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="877a5ecc-1a59-49c9-b78e-968864e2496c" containerName="main"
Jul 28 18:41:39.600445 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.600424    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.603188 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.603164    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"kube-root-ca.crt\""
Jul 28 18:41:39.604269 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.604244    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"default-dockercfg-d9r88\""
Jul 28 18:41:39.604393 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.604245    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"auth-test-model-5523b7-kserve-self-signed-certs\""
Jul 28 18:41:39.604393 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.604241    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"openshift-service-ca.crt\""
Jul 28 18:41:39.609642 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.609619    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"]
Jul 28 18:41:39.765639 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.765584    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tls-certs\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.765908 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.765647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn229\" (UniqueName: \"kubernetes.io/projected/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-kube-api-access-jn229\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.765908 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.765675    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tmp-dir\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.765908 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.765716    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-home\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.765908 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.765747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-model-cache\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.765908 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.765771    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-dshm\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867302 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867206    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tls-certs\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867302 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867252    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn229\" (UniqueName: \"kubernetes.io/projected/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-kube-api-access-jn229\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867302 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tmp-dir\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867302 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867302    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-home\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867655 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-model-cache\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867655 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867345    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-dshm\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867774 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tmp-dir\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867827 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-home\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.867893 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.867843    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-model-cache\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.869724 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.869699    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-dshm\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.869969 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.869951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tls-certs\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.875737 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.875711    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn229\" (UniqueName: \"kubernetes.io/projected/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-kube-api-access-jn229\") pod \"auth-test-model-5523b7-kserve-d7887f7c9-6pmtv\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") " pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:39.911301 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:39.911258    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:40.059692 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:40.059668    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"]
Jul 28 18:41:40.062276 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:41:40.062245    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59451ba5_ab3b_49b2_ba0b_67f72a655aeb.slice/crio-61b6fa9b1340859ef9d4e0959de75db9d54fe2d39e56e85ed6627d4b03a87f32 WatchSource:0}: Error finding container 61b6fa9b1340859ef9d4e0959de75db9d54fe2d39e56e85ed6627d4b03a87f32: Status 404 returned error can't find the container with id 61b6fa9b1340859ef9d4e0959de75db9d54fe2d39e56e85ed6627d4b03a87f32
Jul 28 18:41:40.291530 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:40.291485    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv" event={"ID":"59451ba5-ab3b-49b2-ba0b-67f72a655aeb","Type":"ContainerStarted","Data":"1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b"}
Jul 28 18:41:40.291530 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:40.291532    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv" event={"ID":"59451ba5-ab3b-49b2-ba0b-67f72a655aeb","Type":"ContainerStarted","Data":"61b6fa9b1340859ef9d4e0959de75db9d54fe2d39e56e85ed6627d4b03a87f32"}
Jul 28 18:41:40.311454 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:40.311394    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv" podStartSLOduration=1.311375542 podStartE2EDuration="1.311375542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:41:40.309634007 +0000 UTC m=+2674.455186983" watchObservedRunningTime="2026-07-28 18:41:40.311375542 +0000 UTC m=+2674.456928518"
Jul 28 18:41:43.599589 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:43.599550    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:41:49.911729 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:49.911688    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:49.912178 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:49.911752    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:49.924661 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:49.924632    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:41:50.352293 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:41:50.352261    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:42:07.396255 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:07.396167    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:42:07.404781 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:07.404753    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:42:16.298132 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.298086    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"]
Jul 28 18:42:16.301423 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.301398    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.304141 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.304110    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"openshift-service-ca.crt\""
Jul 28 18:42:16.305012 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.304927    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"kube-root-ca.crt\""
Jul 28 18:42:16.305289 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.305128    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"auth-test-model-10fb8f-kserve-self-signed-certs\""
Jul 28 18:42:16.305289 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.305217    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"default-dockercfg-bqjxr\""
Jul 28 18:42:16.310883 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.310842    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"]
Jul 28 18:42:16.417122 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.417082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tls-certs\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.417122 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.417125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-model-cache\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.417363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.417197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8drl\" (UniqueName: \"kubernetes.io/projected/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-kube-api-access-n8drl\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.417363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.417224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tmp-dir\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.417363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.417253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-home\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.417363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.417277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-dshm\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.518623 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.518571    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tls-certs\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.518623 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.518620    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-model-cache\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.518865 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.518792    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8drl\" (UniqueName: \"kubernetes.io/projected/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-kube-api-access-n8drl\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.518945 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.518879    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tmp-dir\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.519011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.518951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-home\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.519011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.518990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-dshm\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.519123 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.519049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-model-cache\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.519354 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.519334    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-home\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.519463 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.519394    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tmp-dir\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.521118 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.521092    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-dshm\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.521426 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.521402    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tls-certs\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.528694 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.528655    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8drl\" (UniqueName: \"kubernetes.io/projected/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-kube-api-access-n8drl\") pod \"auth-test-model-10fb8f-kserve-69b667889f-wf48b\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") " pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.612992 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.612889    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:16.768993 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:16.768966    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"]
Jul 28 18:42:16.771246 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:42:16.771217    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda10a0a27_ec60_41be_b2ee_23cf8140aa9a.slice/crio-6269ffecfaadb87416d834942161a95b6edd3b576b172a85dd11c0775d236989 WatchSource:0}: Error finding container 6269ffecfaadb87416d834942161a95b6edd3b576b172a85dd11c0775d236989: Status 404 returned error can't find the container with id 6269ffecfaadb87416d834942161a95b6edd3b576b172a85dd11c0775d236989
Jul 28 18:42:17.477027 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:17.476989    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b" event={"ID":"a10a0a27-ec60-41be-b2ee-23cf8140aa9a","Type":"ContainerStarted","Data":"81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426"}
Jul 28 18:42:17.477419 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:17.477039    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b" event={"ID":"a10a0a27-ec60-41be-b2ee-23cf8140aa9a","Type":"ContainerStarted","Data":"6269ffecfaadb87416d834942161a95b6edd3b576b172a85dd11c0775d236989"}
Jul 28 18:42:17.495463 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:17.495411    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b" podStartSLOduration=1.49539483 podStartE2EDuration="1.49539483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:42:17.494510223 +0000 UTC m=+2711.640063196" watchObservedRunningTime="2026-07-28 18:42:17.49539483 +0000 UTC m=+2711.640947803"
Jul 28 18:42:19.895278 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:19.895237    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:42:26.613663 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:26.613624    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:26.614091 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:26.613678    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:26.626670 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:26.626579    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:27.535003 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:27.534968    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:42:46.691407 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:42:46.691364    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:43:07.469635 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:43:07.469604    2575 scope.go:117] "RemoveContainer" containerID="5deec58b48cf22264e28a34c1b26bdec2b9e1b4b68e239fd863a6384915a4bf6"
Jul 28 18:43:07.790761 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:43:07.790723    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:43:26.105284 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:43:26.105246    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:43:53.199197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:43:53.199161    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:44:12.095274 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:44:12.095238    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:44:38.796595 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:44:38.796553    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:44:57.601932 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:44:57.601836    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:45:18.943451 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:45:18.943416    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:45:36.803706 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:45:36.803666    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:46:03.991503 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:46:03.991461    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:46:21.183208 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:46:21.183162    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:46:48.586159 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:46:48.586106    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:46:53.423541 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:46:53.423497    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"]
Jul 28 18:46:54.400006 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:46:54.399980    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"]
Jul 28 18:47:05.593926 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:05.593880    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:47:07.425191 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:07.425160    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:47:07.441526 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:07.441496    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:47:08.443574 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.443534    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv" podUID="59451ba5-ab3b-49b2-ba0b-67f72a655aeb" containerName="main" containerID="cri-o://1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b" gracePeriod=45
Jul 28 18:47:08.703752 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.703677    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:47:08.804803 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.804769    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tmp-dir\") pod \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") "
Jul 28 18:47:08.805029 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.804901    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn229\" (UniqueName: \"kubernetes.io/projected/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-kube-api-access-jn229\") pod \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") "
Jul 28 18:47:08.805029 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.804935    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-dshm\") pod \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") "
Jul 28 18:47:08.805029 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.804973    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-home\") pod \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") "
Jul 28 18:47:08.805201 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805048    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-model-cache\") pod \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") "
Jul 28 18:47:08.805201 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805091    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tls-certs\") pod \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\" (UID: \"59451ba5-ab3b-49b2-ba0b-67f72a655aeb\") "
Jul 28 18:47:08.805201 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805116    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59451ba5-ab3b-49b2-ba0b-67f72a655aeb" (UID: "59451ba5-ab3b-49b2-ba0b-67f72a655aeb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:08.805333 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805247    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-home" (OuterVolumeSpecName: "home") pod "59451ba5-ab3b-49b2-ba0b-67f72a655aeb" (UID: "59451ba5-ab3b-49b2-ba0b-67f72a655aeb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:08.805333 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805299    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-model-cache" (OuterVolumeSpecName: "model-cache") pod "59451ba5-ab3b-49b2-ba0b-67f72a655aeb" (UID: "59451ba5-ab3b-49b2-ba0b-67f72a655aeb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:08.805457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805432    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:08.805457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805451    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:08.805557 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.805459    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:08.807249 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.807206    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-kube-api-access-jn229" (OuterVolumeSpecName: "kube-api-access-jn229") pod "59451ba5-ab3b-49b2-ba0b-67f72a655aeb" (UID: "59451ba5-ab3b-49b2-ba0b-67f72a655aeb"). InnerVolumeSpecName "kube-api-access-jn229". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:47:08.807249 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.807236    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59451ba5-ab3b-49b2-ba0b-67f72a655aeb" (UID: "59451ba5-ab3b-49b2-ba0b-67f72a655aeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:47:08.807417 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.807290    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-dshm" (OuterVolumeSpecName: "dshm") pod "59451ba5-ab3b-49b2-ba0b-67f72a655aeb" (UID: "59451ba5-ab3b-49b2-ba0b-67f72a655aeb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:08.906921 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.906879    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:08.906921 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.906915    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn229\" (UniqueName: \"kubernetes.io/projected/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-kube-api-access-jn229\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:08.906921 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.906928    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59451ba5-ab3b-49b2-ba0b-67f72a655aeb-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:08.940186 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.940154    2575 generic.go:356] "Generic (PLEG): container finished" podID="59451ba5-ab3b-49b2-ba0b-67f72a655aeb" containerID="1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b" exitCode=0
Jul 28 18:47:08.940363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.940198    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"
Jul 28 18:47:08.940363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.940206    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv" event={"ID":"59451ba5-ab3b-49b2-ba0b-67f72a655aeb","Type":"ContainerDied","Data":"1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b"}
Jul 28 18:47:08.940363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.940240    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv" event={"ID":"59451ba5-ab3b-49b2-ba0b-67f72a655aeb","Type":"ContainerDied","Data":"61b6fa9b1340859ef9d4e0959de75db9d54fe2d39e56e85ed6627d4b03a87f32"}
Jul 28 18:47:08.940363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.940257    2575 scope.go:117] "RemoveContainer" containerID="1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b"
Jul 28 18:47:08.950826 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.950803    2575 scope.go:117] "RemoveContainer" containerID="1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b"
Jul 28 18:47:08.951162 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:47:08.951143    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b\": container with ID starting with 1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b not found: ID does not exist" containerID="1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b"
Jul 28 18:47:08.951214 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.951172    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b"} err="failed to get container status \"1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b\": rpc error: code = NotFound desc = could not find container \"1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b\": container with ID starting with 1f4c7994183f1c73006fa954ce23cd046e60b0d7954de927135e26a714ba5c7b not found: ID does not exist"
Jul 28 18:47:08.961744 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.961658    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"]
Jul 28 18:47:08.965905 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:08.965873    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/auth-test-model-5523b7-kserve-d7887f7c9-6pmtv"]
Jul 28 18:47:09.419663 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.419620    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b" podUID="a10a0a27-ec60-41be-b2ee-23cf8140aa9a" containerName="main" containerID="cri-o://81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426" gracePeriod=45
Jul 28 18:47:09.672740 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.672668    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:47:09.816175 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816123    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-model-cache\") pod \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") "
Jul 28 18:47:09.816175 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816188    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8drl\" (UniqueName: \"kubernetes.io/projected/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-kube-api-access-n8drl\") pod \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") "
Jul 28 18:47:09.816453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816220    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tls-certs\") pod \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") "
Jul 28 18:47:09.816453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816236    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tmp-dir\") pod \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") "
Jul 28 18:47:09.816453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816266    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-dshm\") pod \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") "
Jul 28 18:47:09.816453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816294    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-home\") pod \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\" (UID: \"a10a0a27-ec60-41be-b2ee-23cf8140aa9a\") "
Jul 28 18:47:09.816669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816469    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-model-cache" (OuterVolumeSpecName: "model-cache") pod "a10a0a27-ec60-41be-b2ee-23cf8140aa9a" (UID: "a10a0a27-ec60-41be-b2ee-23cf8140aa9a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:09.816669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816556    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a10a0a27-ec60-41be-b2ee-23cf8140aa9a" (UID: "a10a0a27-ec60-41be-b2ee-23cf8140aa9a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:09.816669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.816626    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-home" (OuterVolumeSpecName: "home") pod "a10a0a27-ec60-41be-b2ee-23cf8140aa9a" (UID: "a10a0a27-ec60-41be-b2ee-23cf8140aa9a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:09.818496 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.818471    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-dshm" (OuterVolumeSpecName: "dshm") pod "a10a0a27-ec60-41be-b2ee-23cf8140aa9a" (UID: "a10a0a27-ec60-41be-b2ee-23cf8140aa9a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:47:09.818577 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.818538    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-kube-api-access-n8drl" (OuterVolumeSpecName: "kube-api-access-n8drl") pod "a10a0a27-ec60-41be-b2ee-23cf8140aa9a" (UID: "a10a0a27-ec60-41be-b2ee-23cf8140aa9a"). InnerVolumeSpecName "kube-api-access-n8drl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:47:09.818617 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.818598    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a10a0a27-ec60-41be-b2ee-23cf8140aa9a" (UID: "a10a0a27-ec60-41be-b2ee-23cf8140aa9a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:47:09.917655 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.917623    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:09.917655 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.917650    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:09.917655 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.917663    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8drl\" (UniqueName: \"kubernetes.io/projected/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-kube-api-access-n8drl\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:09.917940 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.917673    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:09.917940 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.917683    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:09.917940 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.917690    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10a0a27-ec60-41be-b2ee-23cf8140aa9a-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:47:09.947268 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.947169    2575 generic.go:356] "Generic (PLEG): container finished" podID="a10a0a27-ec60-41be-b2ee-23cf8140aa9a" containerID="81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426" exitCode=0
Jul 28 18:47:09.947268 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.947232    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"
Jul 28 18:47:09.947496 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.947259    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b" event={"ID":"a10a0a27-ec60-41be-b2ee-23cf8140aa9a","Type":"ContainerDied","Data":"81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426"}
Jul 28 18:47:09.947496 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.947309    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b" event={"ID":"a10a0a27-ec60-41be-b2ee-23cf8140aa9a","Type":"ContainerDied","Data":"6269ffecfaadb87416d834942161a95b6edd3b576b172a85dd11c0775d236989"}
Jul 28 18:47:09.947496 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.947331    2575 scope.go:117] "RemoveContainer" containerID="81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426"
Jul 28 18:47:09.956534 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.956498    2575 scope.go:117] "RemoveContainer" containerID="81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426"
Jul 28 18:47:09.956954 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:47:09.956929    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426\": container with ID starting with 81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426 not found: ID does not exist" containerID="81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426"
Jul 28 18:47:09.957041 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.956963    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426"} err="failed to get container status \"81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426\": rpc error: code = NotFound desc = could not find container \"81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426\": container with ID starting with 81763fc990f51040c1ed2abce215f00ab64ccda4d680271b56a5b818fea5c426 not found: ID does not exist"
Jul 28 18:47:09.971222 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.971186    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"]
Jul 28 18:47:09.974732 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:09.974697    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/auth-test-model-10fb8f-kserve-69b667889f-wf48b"]
Jul 28 18:47:10.400843 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:10.400806    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59451ba5-ab3b-49b2-ba0b-67f72a655aeb" path="/var/lib/kubelet/pods/59451ba5-ab3b-49b2-ba0b-67f72a655aeb/volumes"
Jul 28 18:47:10.401277 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:47:10.401261    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a10a0a27-ec60-41be-b2ee-23cf8140aa9a" path="/var/lib/kubelet/pods/a10a0a27-ec60-41be-b2ee-23cf8140aa9a/volumes"
Jul 28 18:48:07.101744 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.101709    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"]
Jul 28 18:48:07.102333 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.102272    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59451ba5-ab3b-49b2-ba0b-67f72a655aeb" containerName="main"
Jul 28 18:48:07.102333 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.102294    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59451ba5-ab3b-49b2-ba0b-67f72a655aeb" containerName="main"
Jul 28 18:48:07.102333 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.102327    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a10a0a27-ec60-41be-b2ee-23cf8140aa9a" containerName="main"
Jul 28 18:48:07.102506 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.102336    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a10a0a27-ec60-41be-b2ee-23cf8140aa9a" containerName="main"
Jul 28 18:48:07.102506 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.102424    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a10a0a27-ec60-41be-b2ee-23cf8140aa9a" containerName="main"
Jul 28 18:48:07.102506 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.102435    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="59451ba5-ab3b-49b2-ba0b-67f72a655aeb" containerName="main"
Jul 28 18:48:07.105807 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.105780    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.108606 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.108578    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"openshift-service-ca.crt\""
Jul 28 18:48:07.108781 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.108578    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"default-dockercfg-bqjxr\""
Jul 28 18:48:07.109567 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.109547    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"kube-root-ca.crt\""
Jul 28 18:48:07.109685 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.109547    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"sub-test-model-10fb8f-kserve-self-signed-certs\""
Jul 28 18:48:07.117057 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.117024    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"]
Jul 28 18:48:07.226980 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.226937    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqnqx\" (UniqueName: \"kubernetes.io/projected/a29991b1-94d0-4570-a577-72739ee57a7b-kube-api-access-mqnqx\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.227163 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.227010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-model-cache\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.227163 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.227039    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-home\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.227163 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.227083    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29991b1-94d0-4570-a577-72739ee57a7b-tls-certs\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.227163 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.227128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-dshm\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.227163 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.227150    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-tmp-dir\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.327715 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.327672    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-model-cache\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.327935 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.327730    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-home\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.327935 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.327783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29991b1-94d0-4570-a577-72739ee57a7b-tls-certs\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.327935 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.327838    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-dshm\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.327935 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.327911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-tmp-dir\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.328170 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.327943    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqnqx\" (UniqueName: \"kubernetes.io/projected/a29991b1-94d0-4570-a577-72739ee57a7b-kube-api-access-mqnqx\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.328224 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.328207    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-model-cache\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.328588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.328548    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-tmp-dir\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.328588 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.328579    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-home\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.331100 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.331069    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29991b1-94d0-4570-a577-72739ee57a7b-tls-certs\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.331233 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.331130    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-dshm\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.337326 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.337289    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqnqx\" (UniqueName: \"kubernetes.io/projected/a29991b1-94d0-4570-a577-72739ee57a7b-kube-api-access-mqnqx\") pod \"sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") " pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.417939 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.417794    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:07.571441 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:07.571411    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"]
Jul 28 18:48:07.573650 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:48:07.573620    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda29991b1_94d0_4570_a577_72739ee57a7b.slice/crio-3ff11c32374f7b8a55777825a81147af0c87921a4e7c247b40ebf569a6cb608f WatchSource:0}: Error finding container 3ff11c32374f7b8a55777825a81147af0c87921a4e7c247b40ebf569a6cb608f: Status 404 returned error can't find the container with id 3ff11c32374f7b8a55777825a81147af0c87921a4e7c247b40ebf569a6cb608f
Jul 28 18:48:08.242058 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:08.242015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl" event={"ID":"a29991b1-94d0-4570-a577-72739ee57a7b","Type":"ContainerStarted","Data":"8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1"}
Jul 28 18:48:08.242058 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:08.242062    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl" event={"ID":"a29991b1-94d0-4570-a577-72739ee57a7b","Type":"ContainerStarted","Data":"3ff11c32374f7b8a55777825a81147af0c87921a4e7c247b40ebf569a6cb608f"}
Jul 28 18:48:08.292227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:08.292169    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl" podStartSLOduration=1.292150719 podStartE2EDuration="1.292150719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:48:08.288974042 +0000 UTC m=+3062.434527017" watchObservedRunningTime="2026-07-28 18:48:08.292150719 +0000 UTC m=+3062.437703692"
Jul 28 18:48:10.890590 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:10.890550    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:48:17.418481 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:17.418427    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:17.418481 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:17.418501    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:17.431363 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:17.431334    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:18.303355 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:18.303317    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:48:44.386093 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:48:44.386052    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:49:01.895548 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:01.895505    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:49:15.674785 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:15.674746    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"]
Jul 28 18:49:17.582697 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:17.582657    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:49:24.696764 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.696725    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"]
Jul 28 18:49:24.700212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.700191    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.702578 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.702556    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"kube-root-ca.crt\""
Jul 28 18:49:24.702719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.702560    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"rate-test-model-5523b7-kserve-self-signed-certs\""
Jul 28 18:49:24.702719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.702600    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"openshift-service-ca.crt\""
Jul 28 18:49:24.703548 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.703524    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5523b7\"/\"default-dockercfg-d9r88\""
Jul 28 18:49:24.709791 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.709768    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"]
Jul 28 18:49:24.775381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.775348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-442g8\" (UniqueName: \"kubernetes.io/projected/ce5c1563-3296-430a-92ae-4bf542f9830d-kube-api-access-442g8\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.775566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.775398    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-tmp-dir\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.775566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.775418    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce5c1563-3296-430a-92ae-4bf542f9830d-tls-certs\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.775566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.775516    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-home\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.775566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.775545    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-model-cache\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.775736 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.775574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-dshm\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.877114 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877074    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-home\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.877114 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-model-cache\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.877381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-dshm\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.877381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877196    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-442g8\" (UniqueName: \"kubernetes.io/projected/ce5c1563-3296-430a-92ae-4bf542f9830d-kube-api-access-442g8\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.877381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-tmp-dir\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.877381 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce5c1563-3296-430a-92ae-4bf542f9830d-tls-certs\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.881918 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877920    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-tmp-dir\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.881918 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.877959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-home\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.881918 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.878062    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-model-cache\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.881918 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.880966    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce5c1563-3296-430a-92ae-4bf542f9830d-tls-certs\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.886050 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.886020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-dshm\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:24.886635 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:24.886616    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-442g8\" (UniqueName: \"kubernetes.io/projected/ce5c1563-3296-430a-92ae-4bf542f9830d-kube-api-access-442g8\") pod \"rate-test-model-5523b7-kserve-684f4d747-5zrzg\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") " pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:25.010356 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:25.010311    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:25.154764 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:25.154734    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"]
Jul 28 18:49:25.157401 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:49:25.157370    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce5c1563_3296_430a_92ae_4bf542f9830d.slice/crio-f1997eead1ac7922a28c39fff437c87e8933f42c166a39be3250a1dc7cb83983 WatchSource:0}: Error finding container f1997eead1ac7922a28c39fff437c87e8933f42c166a39be3250a1dc7cb83983: Status 404 returned error can't find the container with id f1997eead1ac7922a28c39fff437c87e8933f42c166a39be3250a1dc7cb83983
Jul 28 18:49:25.633566 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:25.633523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg" event={"ID":"ce5c1563-3296-430a-92ae-4bf542f9830d","Type":"ContainerStarted","Data":"be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06"}
Jul 28 18:49:25.633768 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:25.633582    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg" event={"ID":"ce5c1563-3296-430a-92ae-4bf542f9830d","Type":"ContainerStarted","Data":"f1997eead1ac7922a28c39fff437c87e8933f42c166a39be3250a1dc7cb83983"}
Jul 28 18:49:25.651081 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:25.651026    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg" podStartSLOduration=1.6510095470000001 podStartE2EDuration="1.651009547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:49:25.650044724 +0000 UTC m=+3139.795597709" watchObservedRunningTime="2026-07-28 18:49:25.651009547 +0000 UTC m=+3139.796562538"
Jul 28 18:49:30.694423 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:30.694326    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl" podUID="a29991b1-94d0-4570-a577-72739ee57a7b" containerName="main" containerID="cri-o://8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1" gracePeriod=45
Jul 28 18:49:30.959968 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:30.959885    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:49:31.140341 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.140303    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqnqx\" (UniqueName: \"kubernetes.io/projected/a29991b1-94d0-4570-a577-72739ee57a7b-kube-api-access-mqnqx\") pod \"a29991b1-94d0-4570-a577-72739ee57a7b\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") "
Jul 28 18:49:31.140341 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.140342    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-model-cache\") pod \"a29991b1-94d0-4570-a577-72739ee57a7b\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") "
Jul 28 18:49:31.140602 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.140373    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-dshm\") pod \"a29991b1-94d0-4570-a577-72739ee57a7b\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") "
Jul 28 18:49:31.140602 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.140395    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-tmp-dir\") pod \"a29991b1-94d0-4570-a577-72739ee57a7b\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") "
Jul 28 18:49:31.140602 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.140451    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-home\") pod \"a29991b1-94d0-4570-a577-72739ee57a7b\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") "
Jul 28 18:49:31.140602 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.140483    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29991b1-94d0-4570-a577-72739ee57a7b-tls-certs\") pod \"a29991b1-94d0-4570-a577-72739ee57a7b\" (UID: \"a29991b1-94d0-4570-a577-72739ee57a7b\") "
Jul 28 18:49:31.141531 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.141018    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a29991b1-94d0-4570-a577-72739ee57a7b" (UID: "a29991b1-94d0-4570-a577-72739ee57a7b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:31.141531 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.141254    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a29991b1-94d0-4570-a577-72739ee57a7b" (UID: "a29991b1-94d0-4570-a577-72739ee57a7b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:31.141731 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.141566    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-home" (OuterVolumeSpecName: "home") pod "a29991b1-94d0-4570-a577-72739ee57a7b" (UID: "a29991b1-94d0-4570-a577-72739ee57a7b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:31.141731 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.141706    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:49:31.143843 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.143801    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-dshm" (OuterVolumeSpecName: "dshm") pod "a29991b1-94d0-4570-a577-72739ee57a7b" (UID: "a29991b1-94d0-4570-a577-72739ee57a7b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:49:31.149006 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.145603    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a29991b1-94d0-4570-a577-72739ee57a7b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a29991b1-94d0-4570-a577-72739ee57a7b" (UID: "a29991b1-94d0-4570-a577-72739ee57a7b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:49:31.151087 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.149455    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29991b1-94d0-4570-a577-72739ee57a7b-kube-api-access-mqnqx" (OuterVolumeSpecName: "kube-api-access-mqnqx") pod "a29991b1-94d0-4570-a577-72739ee57a7b" (UID: "a29991b1-94d0-4570-a577-72739ee57a7b"). InnerVolumeSpecName "kube-api-access-mqnqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:49:31.242671 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.242558    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqnqx\" (UniqueName: \"kubernetes.io/projected/a29991b1-94d0-4570-a577-72739ee57a7b-kube-api-access-mqnqx\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:49:31.242671 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.242609    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:49:31.242671 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.242621    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:49:31.242671 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.242629    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a29991b1-94d0-4570-a577-72739ee57a7b-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:49:31.242671 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.242637    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a29991b1-94d0-4570-a577-72739ee57a7b-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:49:31.663329 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.663289    2575 generic.go:356] "Generic (PLEG): container finished" podID="a29991b1-94d0-4570-a577-72739ee57a7b" containerID="8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1" exitCode=0
Jul 28 18:49:31.663513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.663345    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"
Jul 28 18:49:31.663513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.663379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl" event={"ID":"a29991b1-94d0-4570-a577-72739ee57a7b","Type":"ContainerDied","Data":"8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1"}
Jul 28 18:49:31.663513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.663424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl" event={"ID":"a29991b1-94d0-4570-a577-72739ee57a7b","Type":"ContainerDied","Data":"3ff11c32374f7b8a55777825a81147af0c87921a4e7c247b40ebf569a6cb608f"}
Jul 28 18:49:31.663513 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.663441    2575 scope.go:117] "RemoveContainer" containerID="8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1"
Jul 28 18:49:31.675898 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.675875    2575 scope.go:117] "RemoveContainer" containerID="8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1"
Jul 28 18:49:31.676245 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:49:31.676225    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1\": container with ID starting with 8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1 not found: ID does not exist" containerID="8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1"
Jul 28 18:49:31.676286 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.676260    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1"} err="failed to get container status \"8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1\": rpc error: code = NotFound desc = could not find container \"8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1\": container with ID starting with 8e29c478c50ae27f2773c33845803eb67dd5c6f232b61ace79c8e8144d5b7bd1 not found: ID does not exist"
Jul 28 18:49:31.686094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.686058    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"]
Jul 28 18:49:31.691547 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:31.691514    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/sub-test-model-10fb8f-kserve-5d487b5f6f-6fqzl"]
Jul 28 18:49:32.401959 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:32.401916    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a29991b1-94d0-4570-a577-72739ee57a7b" path="/var/lib/kubelet/pods/a29991b1-94d0-4570-a577-72739ee57a7b/volumes"
Jul 28 18:49:35.010905 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:35.010844    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:35.011322 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:35.010920    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:35.023639 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:35.023610    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:35.697444 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:35.697411    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:49:36.885544 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:36.885505    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:49:45.888043 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.887998    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"]
Jul 28 18:49:45.888586 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.888418    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29991b1-94d0-4570-a577-72739ee57a7b" containerName="main"
Jul 28 18:49:45.888586 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.888430    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29991b1-94d0-4570-a577-72739ee57a7b" containerName="main"
Jul 28 18:49:45.888586 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.888497    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a29991b1-94d0-4570-a577-72739ee57a7b" containerName="main"
Jul 28 18:49:45.893259 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.893227    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:45.896641 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.896608    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"kube-root-ca.crt\""
Jul 28 18:49:45.896641 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.896608    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"rate-test-model-10fb8f-kserve-self-signed-certs\""
Jul 28 18:49:45.896924 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.896617    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"default-dockercfg-bqjxr\""
Jul 28 18:49:45.897011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.896990    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-10fb8f\"/\"openshift-service-ca.crt\""
Jul 28 18:49:45.907244 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.907213    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"]
Jul 28 18:49:45.973955 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.973916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwq9\" (UniqueName: \"kubernetes.io/projected/b7b05149-0318-45b4-bb7f-42e05f2232e5-kube-api-access-xmwq9\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:45.973955 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.973957    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-tmp-dir\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:45.974212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.973983    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-dshm\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:45.974212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.974062    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b05149-0318-45b4-bb7f-42e05f2232e5-tls-certs\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:45.974212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.974164    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-home\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:45.974212 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:45.974210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-model-cache\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.075592 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.075556    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwq9\" (UniqueName: \"kubernetes.io/projected/b7b05149-0318-45b4-bb7f-42e05f2232e5-kube-api-access-xmwq9\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.075592 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.075599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-tmp-dir\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.075891 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.075729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-dshm\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.075891 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.075774    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b05149-0318-45b4-bb7f-42e05f2232e5-tls-certs\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.075891 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.075843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-home\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.076073 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.075915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-model-cache\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.076073 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.076035    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-tmp-dir\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.076207 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.076178    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-home\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.076337 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.076264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-model-cache\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.078162 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.078138    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-dshm\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.078377 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.078356    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b05149-0318-45b4-bb7f-42e05f2232e5-tls-certs\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.084101 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.084073    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwq9\" (UniqueName: \"kubernetes.io/projected/b7b05149-0318-45b4-bb7f-42e05f2232e5-kube-api-access-xmwq9\") pod \"rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") " pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.206110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.206010    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:46.359194 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.359156    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"]
Jul 28 18:49:46.361702 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:49:46.361665    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7b05149_0318_45b4_bb7f_42e05f2232e5.slice/crio-76e60067a9b9e4329e6dbba02bebc2e9de141c7ee51130b30fb87eda7fcc26a0 WatchSource:0}: Error finding container 76e60067a9b9e4329e6dbba02bebc2e9de141c7ee51130b30fb87eda7fcc26a0: Status 404 returned error can't find the container with id 76e60067a9b9e4329e6dbba02bebc2e9de141c7ee51130b30fb87eda7fcc26a0
Jul 28 18:49:46.742148 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.742112    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm" event={"ID":"b7b05149-0318-45b4-bb7f-42e05f2232e5","Type":"ContainerStarted","Data":"aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167"}
Jul 28 18:49:46.742425 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.742159    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm" event={"ID":"b7b05149-0318-45b4-bb7f-42e05f2232e5","Type":"ContainerStarted","Data":"76e60067a9b9e4329e6dbba02bebc2e9de141c7ee51130b30fb87eda7fcc26a0"}
Jul 28 18:49:46.760835 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:46.760778    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm" podStartSLOduration=1.760758794 podStartE2EDuration="1.760758794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:49:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:49:46.759443124 +0000 UTC m=+3160.904996120" watchObservedRunningTime="2026-07-28 18:49:46.760758794 +0000 UTC m=+3160.906311840"
Jul 28 18:49:49.783661 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:49.783621    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:49:56.206618 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:56.206566    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:56.207061 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:56.206658    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:56.219638 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:56.219605    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:49:56.803388 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:49:56.803357    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:50:19.799701 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:50:19.799659    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:50:34.487086 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:50:34.487045    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:50:51.689480 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:50:51.689435    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:50:56.177917 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:50:56.177879    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"]
Jul 28 18:50:57.110262 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:50:57.110228    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"]
Jul 28 18:51:07.389062 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:07.389026    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:51:11.197395 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.197348    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg" podUID="ce5c1563-3296-430a-92ae-4bf542f9830d" containerName="main" containerID="cri-o://be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06" gracePeriod=45
Jul 28 18:51:11.457323 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.457250    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:51:11.495146 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495106    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-442g8\" (UniqueName: \"kubernetes.io/projected/ce5c1563-3296-430a-92ae-4bf542f9830d-kube-api-access-442g8\") pod \"ce5c1563-3296-430a-92ae-4bf542f9830d\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") "
Jul 28 18:51:11.495146 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495153    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-model-cache\") pod \"ce5c1563-3296-430a-92ae-4bf542f9830d\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") "
Jul 28 18:51:11.495408 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495212    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-home\") pod \"ce5c1563-3296-430a-92ae-4bf542f9830d\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") "
Jul 28 18:51:11.495408 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495294    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-dshm\") pod \"ce5c1563-3296-430a-92ae-4bf542f9830d\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") "
Jul 28 18:51:11.495408 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495398    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce5c1563-3296-430a-92ae-4bf542f9830d-tls-certs\") pod \"ce5c1563-3296-430a-92ae-4bf542f9830d\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") "
Jul 28 18:51:11.496540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495424    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-tmp-dir\") pod \"ce5c1563-3296-430a-92ae-4bf542f9830d\" (UID: \"ce5c1563-3296-430a-92ae-4bf542f9830d\") "
Jul 28 18:51:11.496540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495466    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce5c1563-3296-430a-92ae-4bf542f9830d" (UID: "ce5c1563-3296-430a-92ae-4bf542f9830d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:11.496540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495478    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-home" (OuterVolumeSpecName: "home") pod "ce5c1563-3296-430a-92ae-4bf542f9830d" (UID: "ce5c1563-3296-430a-92ae-4bf542f9830d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:11.496540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495742    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:11.496540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495761    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:11.496540 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.495809    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce5c1563-3296-430a-92ae-4bf542f9830d" (UID: "ce5c1563-3296-430a-92ae-4bf542f9830d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:11.497635 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.497607    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-dshm" (OuterVolumeSpecName: "dshm") pod "ce5c1563-3296-430a-92ae-4bf542f9830d" (UID: "ce5c1563-3296-430a-92ae-4bf542f9830d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:11.497807 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.497786    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce5c1563-3296-430a-92ae-4bf542f9830d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce5c1563-3296-430a-92ae-4bf542f9830d" (UID: "ce5c1563-3296-430a-92ae-4bf542f9830d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:51:11.497903 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.497814    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce5c1563-3296-430a-92ae-4bf542f9830d-kube-api-access-442g8" (OuterVolumeSpecName: "kube-api-access-442g8") pod "ce5c1563-3296-430a-92ae-4bf542f9830d" (UID: "ce5c1563-3296-430a-92ae-4bf542f9830d"). InnerVolumeSpecName "kube-api-access-442g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:51:11.597306 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.597262    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce5c1563-3296-430a-92ae-4bf542f9830d-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:11.597306 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.597297    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:11.597306 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.597307    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-442g8\" (UniqueName: \"kubernetes.io/projected/ce5c1563-3296-430a-92ae-4bf542f9830d-kube-api-access-442g8\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:11.597543 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:11.597318    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce5c1563-3296-430a-92ae-4bf542f9830d-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:12.131377 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.131328    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm" podUID="b7b05149-0318-45b4-bb7f-42e05f2232e5" containerName="main" containerID="cri-o://aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167" gracePeriod=45
Jul 28 18:51:12.177952 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.177913    2575 generic.go:356] "Generic (PLEG): container finished" podID="ce5c1563-3296-430a-92ae-4bf542f9830d" containerID="be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06" exitCode=0
Jul 28 18:51:12.178131 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.177961    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"
Jul 28 18:51:12.178131 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.177971    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg" event={"ID":"ce5c1563-3296-430a-92ae-4bf542f9830d","Type":"ContainerDied","Data":"be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06"}
Jul 28 18:51:12.178131 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.178021    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg" event={"ID":"ce5c1563-3296-430a-92ae-4bf542f9830d","Type":"ContainerDied","Data":"f1997eead1ac7922a28c39fff437c87e8933f42c166a39be3250a1dc7cb83983"}
Jul 28 18:51:12.178131 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.178039    2575 scope.go:117] "RemoveContainer" containerID="be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06"
Jul 28 18:51:12.203164 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.203123    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"]
Jul 28 18:51:12.210303 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.210266    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5523b7/rate-test-model-5523b7-kserve-684f4d747-5zrzg"]
Jul 28 18:51:12.245582 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.244299    2575 scope.go:117] "RemoveContainer" containerID="be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06"
Jul 28 18:51:12.245582 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:51:12.245224    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06\": container with ID starting with be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06 not found: ID does not exist" containerID="be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06"
Jul 28 18:51:12.245582 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.245267    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06"} err="failed to get container status \"be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06\": rpc error: code = NotFound desc = could not find container \"be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06\": container with ID starting with be87523c3fa26abe4893ff94bfa60eb99022137e9a5828c1a824731e6c20ae06 not found: ID does not exist"
Jul 28 18:51:12.397408 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.397333    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:51:12.400126 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.400097    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce5c1563-3296-430a-92ae-4bf542f9830d" path="/var/lib/kubelet/pods/ce5c1563-3296-430a-92ae-4bf542f9830d/volumes"
Jul 28 18:51:12.505319 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505270    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-tmp-dir\") pod \"b7b05149-0318-45b4-bb7f-42e05f2232e5\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") "
Jul 28 18:51:12.505473 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505332    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmwq9\" (UniqueName: \"kubernetes.io/projected/b7b05149-0318-45b4-bb7f-42e05f2232e5-kube-api-access-xmwq9\") pod \"b7b05149-0318-45b4-bb7f-42e05f2232e5\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") "
Jul 28 18:51:12.505473 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505381    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-dshm\") pod \"b7b05149-0318-45b4-bb7f-42e05f2232e5\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") "
Jul 28 18:51:12.505473 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505405    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-home\") pod \"b7b05149-0318-45b4-bb7f-42e05f2232e5\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") "
Jul 28 18:51:12.505473 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505433    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-model-cache\") pod \"b7b05149-0318-45b4-bb7f-42e05f2232e5\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") "
Jul 28 18:51:12.505473 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505461    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b05149-0318-45b4-bb7f-42e05f2232e5-tls-certs\") pod \"b7b05149-0318-45b4-bb7f-42e05f2232e5\" (UID: \"b7b05149-0318-45b4-bb7f-42e05f2232e5\") "
Jul 28 18:51:12.505770 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505608    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7b05149-0318-45b4-bb7f-42e05f2232e5" (UID: "b7b05149-0318-45b4-bb7f-42e05f2232e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:12.505770 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505711    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-home" (OuterVolumeSpecName: "home") pod "b7b05149-0318-45b4-bb7f-42e05f2232e5" (UID: "b7b05149-0318-45b4-bb7f-42e05f2232e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:12.505770 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505728    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7b05149-0318-45b4-bb7f-42e05f2232e5" (UID: "b7b05149-0318-45b4-bb7f-42e05f2232e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:12.505949 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505811    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:12.505949 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505831    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:12.505949 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.505844    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:12.507551 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.507522    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7b05149-0318-45b4-bb7f-42e05f2232e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7b05149-0318-45b4-bb7f-42e05f2232e5" (UID: "b7b05149-0318-45b4-bb7f-42e05f2232e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:51:12.507551 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.507546    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-dshm" (OuterVolumeSpecName: "dshm") pod "b7b05149-0318-45b4-bb7f-42e05f2232e5" (UID: "b7b05149-0318-45b4-bb7f-42e05f2232e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:51:12.507697 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.507555    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7b05149-0318-45b4-bb7f-42e05f2232e5-kube-api-access-xmwq9" (OuterVolumeSpecName: "kube-api-access-xmwq9") pod "b7b05149-0318-45b4-bb7f-42e05f2232e5" (UID: "b7b05149-0318-45b4-bb7f-42e05f2232e5"). InnerVolumeSpecName "kube-api-access-xmwq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:51:12.606950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.606908    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmwq9\" (UniqueName: \"kubernetes.io/projected/b7b05149-0318-45b4-bb7f-42e05f2232e5-kube-api-access-xmwq9\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:12.606950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.606941    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b05149-0318-45b4-bb7f-42e05f2232e5-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:12.606950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:12.606951    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b05149-0318-45b4-bb7f-42e05f2232e5-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:51:13.185910 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.185875    2575 generic.go:356] "Generic (PLEG): container finished" podID="b7b05149-0318-45b4-bb7f-42e05f2232e5" containerID="aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167" exitCode=0
Jul 28 18:51:13.186109 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.185924    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"
Jul 28 18:51:13.186109 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.185965    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm" event={"ID":"b7b05149-0318-45b4-bb7f-42e05f2232e5","Type":"ContainerDied","Data":"aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167"}
Jul 28 18:51:13.186109 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.186012    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm" event={"ID":"b7b05149-0318-45b4-bb7f-42e05f2232e5","Type":"ContainerDied","Data":"76e60067a9b9e4329e6dbba02bebc2e9de141c7ee51130b30fb87eda7fcc26a0"}
Jul 28 18:51:13.186109 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.186033    2575 scope.go:117] "RemoveContainer" containerID="aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167"
Jul 28 18:51:13.195521 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.195504    2575 scope.go:117] "RemoveContainer" containerID="aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167"
Jul 28 18:51:13.195833 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:51:13.195809    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167\": container with ID starting with aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167 not found: ID does not exist" containerID="aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167"
Jul 28 18:51:13.195979 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.195838    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167"} err="failed to get container status \"aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167\": rpc error: code = NotFound desc = could not find container \"aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167\": container with ID starting with aa3ee6c707a3bb3c2d041f91e764abd506d7c8e140d24c797638aa0aa082a167 not found: ID does not exist"
Jul 28 18:51:13.207048 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.207013    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"]
Jul 28 18:51:13.211301 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:13.211273    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-10fb8f/rate-test-model-10fb8f-kserve-5f7f67cb75-tljgm"]
Jul 28 18:51:14.400018 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:14.399982    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7b05149-0318-45b4-bb7f-42e05f2232e5" path="/var/lib/kubelet/pods/b7b05149-0318-45b4-bb7f-42e05f2232e5/volumes"
Jul 28 18:51:29.337529 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:29.337490    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:51:46.316110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.316074    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"]
Jul 28 18:51:46.316599 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.316585    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7b05149-0318-45b4-bb7f-42e05f2232e5" containerName="main"
Jul 28 18:51:46.316648 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.316603    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7b05149-0318-45b4-bb7f-42e05f2232e5" containerName="main"
Jul 28 18:51:46.316648 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.316635    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce5c1563-3296-430a-92ae-4bf542f9830d" containerName="main"
Jul 28 18:51:46.316648 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.316644    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce5c1563-3296-430a-92ae-4bf542f9830d" containerName="main"
Jul 28 18:51:46.316759 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.316747    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7b05149-0318-45b4-bb7f-42e05f2232e5" containerName="main"
Jul 28 18:51:46.316795 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.316764    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce5c1563-3296-430a-92ae-4bf542f9830d" containerName="main"
Jul 28 18:51:46.320191 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.320169    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.322636 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.322613    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-5b063e-openshift-default-dockercfg-km5gw\""
Jul 28 18:51:46.323356 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323330    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-socket\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323454 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323367    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-credential-socket\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323454 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323429    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-data\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323527 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323485    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e044b56b-d35a-4b02-825b-ac00076f514e-istio-podinfo\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323527 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26mr7\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-kube-api-access-26mr7\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323647 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-istio-token\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323706 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-envoy\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323763 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e044b56b-d35a-4b02-825b-ac00076f514e-istiod-ca-cert\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.323820 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.323757    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-certs\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.332245 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.332207    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"]
Jul 28 18:51:46.424672 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-socket\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424672 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-credential-socket\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424982 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424709    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-data\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424982 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424741    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e044b56b-d35a-4b02-825b-ac00076f514e-istio-podinfo\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424982 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424767    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26mr7\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-kube-api-access-26mr7\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424982 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-istio-token\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424982 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424882    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-envoy\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424982 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e044b56b-d35a-4b02-825b-ac00076f514e-istiod-ca-cert\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.424982 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.424944    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-certs\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.425323 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.425189    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-socket\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.425323 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.425208    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-credential-socket\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.425666 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.425632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-certs\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.425867 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.425638    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-data\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.426127 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.426102    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e044b56b-d35a-4b02-825b-ac00076f514e-istiod-ca-cert\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.427610 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.427585    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-envoy\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.427999 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.427980    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e044b56b-d35a-4b02-825b-ac00076f514e-istio-podinfo\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.435971 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.435903    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-istio-token\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.436376 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.436347    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26mr7\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-kube-api-access-26mr7\") pod \"e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") " pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.633807 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.633703    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:46.789406 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.789378    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"]
Jul 28 18:51:46.791906 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:51:46.791877    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode044b56b_d35a_4b02_825b_ac00076f514e.slice/crio-23ad5482fcf8c9a1025675f2f065cf760ca487fa1738b654475bba401517727e WatchSource:0}: Error finding container 23ad5482fcf8c9a1025675f2f065cf760ca487fa1738b654475bba401517727e: Status 404 returned error can't find the container with id 23ad5482fcf8c9a1025675f2f065cf760ca487fa1738b654475bba401517727e
Jul 28 18:51:46.794575 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.794542    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:51:46.794656 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.794607    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:51:46.794656 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:46.794637    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:51:47.358659 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:47.358623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" event={"ID":"e044b56b-d35a-4b02-825b-ac00076f514e","Type":"ContainerStarted","Data":"974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f"}
Jul 28 18:51:47.359092 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:47.358677    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" event={"ID":"e044b56b-d35a-4b02-825b-ac00076f514e","Type":"ContainerStarted","Data":"23ad5482fcf8c9a1025675f2f065cf760ca487fa1738b654475bba401517727e"}
Jul 28 18:51:47.386237 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:47.386180    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" podStartSLOduration=1.386162806 podStartE2EDuration="1.386162806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:51:47.385176025 +0000 UTC m=+3281.530728998" watchObservedRunningTime="2026-07-28 18:51:47.386162806 +0000 UTC m=+3281.531715779"
Jul 28 18:51:47.634656 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:47.634552    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:47.640642 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:47.640610    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:48.364810 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:48.364779    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:48.365793 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:48.365773    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:51:59.291901 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:51:59.291846    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:52:07.459799 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.459681    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:52:07.465801 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.465758    2575 generic.go:356] "Generic (PLEG): container finished" podID="e044b56b-d35a-4b02-825b-ac00076f514e" containerID="974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f" exitCode=0
Jul 28 18:52:07.466018 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.465810    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" event={"ID":"e044b56b-d35a-4b02-825b-ac00076f514e","Type":"ContainerDied","Data":"974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f"}
Jul 28 18:52:07.466314 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.466294    2575 scope.go:117] "RemoveContainer" containerID="974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f"
Jul 28 18:52:07.467571 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.467537    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:52:07.467663 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.467621    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:52:07.467663 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.467655    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:52:07.472545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:07.472522    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:52:08.473055 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:08.473022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" event={"ID":"e044b56b-d35a-4b02-825b-ac00076f514e","Type":"ContainerStarted","Data":"4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080"}
Jul 28 18:52:08.635018 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:08.634972    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 18:52:08.635243 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:08.635041    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 18:52:08.635438 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:08.635352    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:52:09.634763 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:09.634713    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 18:52:09.635340 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:09.634782    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 18:52:10.634623 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:10.634571    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 18:52:10.634879 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:10.634664    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 18:52:11.635203 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:11.635155    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 18:52:11.635731 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:11.635229    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 18:52:12.638453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:12.638419    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:52:12.639031 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:12.638724    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:52:12.639662 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:12.639635    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:52:35.695451 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.695358    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"]
Jul 28 18:52:35.699768 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.699733    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.702361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.702319    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-473596\"/\"default-dockercfg-pjtzk\""
Jul 28 18:52:35.703403 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.703289    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-473596\"/\"kube-root-ca.crt\""
Jul 28 18:52:35.703603 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.703587    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-473596\"/\"ipp-route-473596-kserve-self-signed-certs\""
Jul 28 18:52:35.704951 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.704923    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-473596\"/\"openshift-service-ca.crt\""
Jul 28 18:52:35.707177 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.707153    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"]
Jul 28 18:52:35.786335 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.786293    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-home\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.786335 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.786340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-tmp-dir\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.786627 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.786367    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dc6f\" (UniqueName: \"kubernetes.io/projected/a2fe960b-631b-4bba-8972-3eb179db4e43-kube-api-access-7dc6f\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.786627 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.786453    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-model-cache\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.786627 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.786552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fe960b-631b-4bba-8972-3eb179db4e43-tls-certs\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.786627 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.786583    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-dshm\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.887935 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.887895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-home\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888132 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888035    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-tmp-dir\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888132 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dc6f\" (UniqueName: \"kubernetes.io/projected/a2fe960b-631b-4bba-8972-3eb179db4e43-kube-api-access-7dc6f\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888132 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-model-cache\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888312 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fe960b-631b-4bba-8972-3eb179db4e43-tls-certs\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888312 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888210    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-dshm\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888312 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888294    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-home\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888471 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888413    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-tmp-dir\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.888604 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.888580    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-model-cache\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.890635 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.890607    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-dshm\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.890782 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.890765    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fe960b-631b-4bba-8972-3eb179db4e43-tls-certs\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:35.897420 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:35.897392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dc6f\" (UniqueName: \"kubernetes.io/projected/a2fe960b-631b-4bba-8972-3eb179db4e43-kube-api-access-7dc6f\") pod \"ipp-route-473596-kserve-79fdb976f5-58bjn\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") " pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:36.013017 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:36.012982    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:36.167606 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:36.167577    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"]
Jul 28 18:52:36.170023 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:52:36.169990    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2fe960b_631b_4bba_8972_3eb179db4e43.slice/crio-93ab549f8e41601c127024c2892d85a96846212e7e333ffe1e153119d0299dbc WatchSource:0}: Error finding container 93ab549f8e41601c127024c2892d85a96846212e7e333ffe1e153119d0299dbc: Status 404 returned error can't find the container with id 93ab549f8e41601c127024c2892d85a96846212e7e333ffe1e153119d0299dbc
Jul 28 18:52:36.611065 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:36.611028    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn" event={"ID":"a2fe960b-631b-4bba-8972-3eb179db4e43","Type":"ContainerStarted","Data":"fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1"}
Jul 28 18:52:36.611269 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:36.611081    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn" event={"ID":"a2fe960b-631b-4bba-8972-3eb179db4e43","Type":"ContainerStarted","Data":"93ab549f8e41601c127024c2892d85a96846212e7e333ffe1e153119d0299dbc"}
Jul 28 18:52:39.087041 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:39.086981    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn" podStartSLOduration=4.086960299 podStartE2EDuration="4.086960299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:52:36.643896516 +0000 UTC m=+3330.789449490" watchObservedRunningTime="2026-07-28 18:52:39.086960299 +0000 UTC m=+3333.232513272"
Jul 28 18:52:39.087542 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:39.087467    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:52:46.013370 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:46.013326    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:46.013816 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:46.013389    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:46.026439 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:46.026409    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:52:46.674306 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:52:46.674273    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:53:05.989391 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:05.989347    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:53:17.347268 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.347233    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"]
Jul 28 18:53:17.351197 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.351174    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.353723 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.353699    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-94ebbf-openshift-default-dockercfg-cz5kp\""
Jul 28 18:53:17.364476 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.364429    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"]
Jul 28 18:53:17.465632 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.465589    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-envoy\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.465632 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.465635    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf9q6\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-kube-api-access-jf9q6\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.465950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.465698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-data\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.465950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.465736    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09689620-2578-4dda-95c4-19f9da87a700-istio-podinfo\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.465950 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.465779    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-credential-socket\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.466110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.465950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-socket\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.466110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.466002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09689620-2578-4dda-95c4-19f9da87a700-istiod-ca-cert\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.466110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.466070    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-certs\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.466110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.466099    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-istio-token\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567270 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567227    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-envoy\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567270 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf9q6\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-kube-api-access-jf9q6\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-data\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09689620-2578-4dda-95c4-19f9da87a700-istio-podinfo\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-credential-socket\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-socket\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09689620-2578-4dda-95c4-19f9da87a700-istiod-ca-cert\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567497    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-certs\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567545 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-istio-token\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.567937 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-credential-socket\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.568001 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567930    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-data\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.568001 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-certs\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.568001 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.567983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-socket\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.568361 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.568341    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09689620-2578-4dda-95c4-19f9da87a700-istiod-ca-cert\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.569660 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.569637    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-envoy\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.569998 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.569980    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09689620-2578-4dda-95c4-19f9da87a700-istio-podinfo\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.576524 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.576493    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-istio-token\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.576823 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.576797    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf9q6\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-kube-api-access-jf9q6\") pod \"e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") " pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.661873 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.661760    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:17.816957 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.816927    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"]
Jul 28 18:53:17.818819 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:53:17.818773    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09689620_2578_4dda_95c4_19f9da87a700.slice/crio-585e82728135611a2ae96526982acd34bb4c35c3b633f28dc4e92e8e43682545 WatchSource:0}: Error finding container 585e82728135611a2ae96526982acd34bb4c35c3b633f28dc4e92e8e43682545: Status 404 returned error can't find the container with id 585e82728135611a2ae96526982acd34bb4c35c3b633f28dc4e92e8e43682545
Jul 28 18:53:17.821185 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.821149    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:53:17.821310 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.821248    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:53:17.821310 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:17.821291    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:53:18.830761 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:18.830718    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" event={"ID":"09689620-2578-4dda-95c4-19f9da87a700","Type":"ContainerStarted","Data":"1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7"}
Jul 28 18:53:18.831230 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:18.830786    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" event={"ID":"09689620-2578-4dda-95c4-19f9da87a700","Type":"ContainerStarted","Data":"585e82728135611a2ae96526982acd34bb4c35c3b633f28dc4e92e8e43682545"}
Jul 28 18:53:18.853814 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:18.853756    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" podStartSLOduration=1.853737078 podStartE2EDuration="1.853737078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:53:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:53:18.851912229 +0000 UTC m=+3372.997465202" watchObservedRunningTime="2026-07-28 18:53:18.853737078 +0000 UTC m=+3372.999290051"
Jul 28 18:53:19.662825 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:19.662781    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:19.668225 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:19.668194    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:19.838153 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:19.838115    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:19.839200 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:19.839176    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:26.392219 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:26.392171    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:53:34.839677 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.839637    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 28 18:53:34.840230 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.839714    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:53:34.917469 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.917430    2575 generic.go:356] "Generic (PLEG): container finished" podID="09689620-2578-4dda-95c4-19f9da87a700" containerID="1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7" exitCode=0
Jul 28 18:53:34.917644 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.917506    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" event={"ID":"09689620-2578-4dda-95c4-19f9da87a700","Type":"ContainerDied","Data":"1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7"}
Jul 28 18:53:34.918028 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.918009    2575 scope.go:117] "RemoveContainer" containerID="1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7"
Jul 28 18:53:34.919120 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.919094    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:53:34.919183 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.919162    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:53:34.919221 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:34.919194    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 18:53:35.925110 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:35.925071    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" event={"ID":"09689620-2578-4dda-95c4-19f9da87a700","Type":"ContainerStarted","Data":"03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01"}
Jul 28 18:53:36.662001 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:36.661959    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:36.662335 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:36.662313    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 28 18:53:36.662393 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:36.662361    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:53:37.662493 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:37.662447    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 28 18:53:37.663033 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:37.662515    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:53:38.662478 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:38.662440    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 28 18:53:38.662662 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:38.662502    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:53:39.663195 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:39.663154    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 28 18:53:39.663599 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:39.663218    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 28 18:53:40.666312 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:40.666280    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:40.666783 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:40.666668    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:40.667384 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:40.667364    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:53:55.391579 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:55.391542    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:53:58.724310 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:58.724076    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"]
Jul 28 18:53:58.725237 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:53:58.725176    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy" containerID="cri-o://03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01" gracePeriod=30
Jul 28 18:54:03.995183 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:03.995158    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:54:04.086456 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.086420    2575 generic.go:356] "Generic (PLEG): container finished" podID="09689620-2578-4dda-95c4-19f9da87a700" containerID="03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01" exitCode=0
Jul 28 18:54:04.086657 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.086480    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"
Jul 28 18:54:04.086657 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.086504    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" event={"ID":"09689620-2578-4dda-95c4-19f9da87a700","Type":"ContainerDied","Data":"03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01"}
Jul 28 18:54:04.086657 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.086562    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m" event={"ID":"09689620-2578-4dda-95c4-19f9da87a700","Type":"ContainerDied","Data":"585e82728135611a2ae96526982acd34bb4c35c3b633f28dc4e92e8e43682545"}
Jul 28 18:54:04.086657 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.086586    2575 scope.go:117] "RemoveContainer" containerID="03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01"
Jul 28 18:54:04.096256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.096236    2575 scope.go:117] "RemoveContainer" containerID="1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7"
Jul 28 18:54:04.107507 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.106715    2575 scope.go:117] "RemoveContainer" containerID="03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01"
Jul 28 18:54:04.107507 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:54:04.107186    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01\": container with ID starting with 03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01 not found: ID does not exist" containerID="03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01"
Jul 28 18:54:04.107507 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.107226    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01"} err="failed to get container status \"03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01\": rpc error: code = NotFound desc = could not find container \"03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01\": container with ID starting with 03f879e420147f23423ac5c6cec566c6ac07c3af7b90f29e05c9ce292b2cdf01 not found: ID does not exist"
Jul 28 18:54:04.107507 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.107252    2575 scope.go:117] "RemoveContainer" containerID="1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7"
Jul 28 18:54:04.107705 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:54:04.107667    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7\": container with ID starting with 1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7 not found: ID does not exist" containerID="1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7"
Jul 28 18:54:04.107705 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.107694    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7"} err="failed to get container status \"1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7\": rpc error: code = NotFound desc = could not find container \"1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7\": container with ID starting with 1de997fb45021ca1d2142b8a15a020f53bf0fb1f444d4cc096ba04a80aec4cc7 not found: ID does not exist"
Jul 28 18:54:04.139065 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139030    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09689620-2578-4dda-95c4-19f9da87a700-istiod-ca-cert\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139098    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-certs\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139118    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-socket\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139144    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-credential-socket\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139187    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-envoy\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139205    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-istio-token\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139256 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139230    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf9q6\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-kube-api-access-jf9q6\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139558 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139263    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-data\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139558 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09689620-2578-4dda-95c4-19f9da87a700-istio-podinfo\") pod \"09689620-2578-4dda-95c4-19f9da87a700\" (UID: \"09689620-2578-4dda-95c4-19f9da87a700\") "
Jul 28 18:54:04.139558 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139537    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:04.139710 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139553    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:04.139710 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139587    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09689620-2578-4dda-95c4-19f9da87a700-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:54:04.139812 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139707    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.139812 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139725    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-credential-socket\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.139812 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139808    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09689620-2578-4dda-95c4-19f9da87a700-istiod-ca-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.139980 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.139794    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:04.140547 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.140522    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-data" (OuterVolumeSpecName: "istio-data") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:04.141764 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.141737    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-kube-api-access-jf9q6" (OuterVolumeSpecName: "kube-api-access-jf9q6") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "kube-api-access-jf9q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:04.141863 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.141746    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-istio-token" (OuterVolumeSpecName: "istio-token") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:04.142036 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.142015    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:04.142076 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.142037    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09689620-2578-4dda-95c4-19f9da87a700-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09689620-2578-4dda-95c4-19f9da87a700" (UID: "09689620-2578-4dda-95c4-19f9da87a700"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:54:04.240886 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.240809    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09689620-2578-4dda-95c4-19f9da87a700-istio-podinfo\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.240886 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.240847    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-workload-socket\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.240886 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.240894    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-envoy\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.240886 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.240902    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-istio-token\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.241186 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.240911    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf9q6\" (UniqueName: \"kubernetes.io/projected/09689620-2578-4dda-95c4-19f9da87a700-kube-api-access-jf9q6\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.241186 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.240921    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09689620-2578-4dda-95c4-19f9da87a700-istio-data\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:04.410952 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.410912    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"]
Jul 28 18:54:04.421605 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:04.417369    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-94ebbf-openshift-default-547d6647fb-n582m"]
Jul 28 18:54:06.401270 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:06.401240    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09689620-2578-4dda-95c4-19f9da87a700" path="/var/lib/kubelet/pods/09689620-2578-4dda-95c4-19f9da87a700/volumes"
Jul 28 18:54:19.088154 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:19.088115    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:54:23.802394 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:23.802357    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"]
Jul 28 18:54:29.793773 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:29.793735    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:54:38.822299 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:38.822236    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn" podUID="a2fe960b-631b-4bba-8972-3eb179db4e43" containerName="main" containerID="cri-o://fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1" gracePeriod=45
Jul 28 18:54:39.082080 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.082005    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:54:39.163891 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.163834    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-home\") pod \"a2fe960b-631b-4bba-8972-3eb179db4e43\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") "
Jul 28 18:54:39.164094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.163910    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fe960b-631b-4bba-8972-3eb179db4e43-tls-certs\") pod \"a2fe960b-631b-4bba-8972-3eb179db4e43\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") "
Jul 28 18:54:39.164094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.163969    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-dshm\") pod \"a2fe960b-631b-4bba-8972-3eb179db4e43\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") "
Jul 28 18:54:39.164094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164003    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-model-cache\") pod \"a2fe960b-631b-4bba-8972-3eb179db4e43\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") "
Jul 28 18:54:39.164094 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164043    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dc6f\" (UniqueName: \"kubernetes.io/projected/a2fe960b-631b-4bba-8972-3eb179db4e43-kube-api-access-7dc6f\") pod \"a2fe960b-631b-4bba-8972-3eb179db4e43\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") "
Jul 28 18:54:39.164314 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164101    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-tmp-dir\") pod \"a2fe960b-631b-4bba-8972-3eb179db4e43\" (UID: \"a2fe960b-631b-4bba-8972-3eb179db4e43\") "
Jul 28 18:54:39.164314 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164147    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-home" (OuterVolumeSpecName: "home") pod "a2fe960b-631b-4bba-8972-3eb179db4e43" (UID: "a2fe960b-631b-4bba-8972-3eb179db4e43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:39.164314 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164287    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2fe960b-631b-4bba-8972-3eb179db4e43" (UID: "a2fe960b-631b-4bba-8972-3eb179db4e43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:39.164464 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164439    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2fe960b-631b-4bba-8972-3eb179db4e43" (UID: "a2fe960b-631b-4bba-8972-3eb179db4e43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:39.164506 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164460    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-home\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:39.164506 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.164478    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-model-cache\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:39.166311 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.166277    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-dshm" (OuterVolumeSpecName: "dshm") pod "a2fe960b-631b-4bba-8972-3eb179db4e43" (UID: "a2fe960b-631b-4bba-8972-3eb179db4e43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:54:39.166457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.166338    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fe960b-631b-4bba-8972-3eb179db4e43-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2fe960b-631b-4bba-8972-3eb179db4e43" (UID: "a2fe960b-631b-4bba-8972-3eb179db4e43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:54:39.166457 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.166360    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2fe960b-631b-4bba-8972-3eb179db4e43-kube-api-access-7dc6f" (OuterVolumeSpecName: "kube-api-access-7dc6f") pod "a2fe960b-631b-4bba-8972-3eb179db4e43" (UID: "a2fe960b-631b-4bba-8972-3eb179db4e43"). InnerVolumeSpecName "kube-api-access-7dc6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:39.265048 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.265013    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2fe960b-631b-4bba-8972-3eb179db4e43-tls-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:39.265048 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.265044    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-dshm\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:39.265048 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.265053    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dc6f\" (UniqueName: \"kubernetes.io/projected/a2fe960b-631b-4bba-8972-3eb179db4e43-kube-api-access-7dc6f\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:39.265311 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.265084    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2fe960b-631b-4bba-8972-3eb179db4e43-tmp-dir\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:54:39.272754 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.272713    2575 generic.go:356] "Generic (PLEG): container finished" podID="a2fe960b-631b-4bba-8972-3eb179db4e43" containerID="fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1" exitCode=0
Jul 28 18:54:39.272977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.272764    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"
Jul 28 18:54:39.272977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.272801    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn" event={"ID":"a2fe960b-631b-4bba-8972-3eb179db4e43","Type":"ContainerDied","Data":"fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1"}
Jul 28 18:54:39.272977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.272848    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn" event={"ID":"a2fe960b-631b-4bba-8972-3eb179db4e43","Type":"ContainerDied","Data":"93ab549f8e41601c127024c2892d85a96846212e7e333ffe1e153119d0299dbc"}
Jul 28 18:54:39.272977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.272893    2575 scope.go:117] "RemoveContainer" containerID="fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1"
Jul 28 18:54:39.282661 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.282636    2575 scope.go:117] "RemoveContainer" containerID="fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1"
Jul 28 18:54:39.283015 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:54:39.282993    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1\": container with ID starting with fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1 not found: ID does not exist" containerID="fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1"
Jul 28 18:54:39.283103 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.283029    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1"} err="failed to get container status \"fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1\": rpc error: code = NotFound desc = could not find container \"fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1\": container with ID starting with fa4d26c48bf41dc6dfb1dd2e57525a1314a350b050b1237a7578a35c10ec67f1 not found: ID does not exist"
Jul 28 18:54:39.294306 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.294270    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"]
Jul 28 18:54:39.298230 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:39.298196    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-473596/ipp-route-473596-kserve-79fdb976f5-58bjn"]
Jul 28 18:54:40.401046 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:54:40.401011    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2fe960b-631b-4bba-8972-3eb179db4e43" path="/var/lib/kubelet/pods/a2fe960b-631b-4bba-8972-3eb179db4e43/volumes"
Jul 28 18:55:07.537766 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.537732    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"]
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538112    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2fe960b-631b-4bba-8972-3eb179db4e43" containerName="main"
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538123    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fe960b-631b-4bba-8972-3eb179db4e43" containerName="main"
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538139    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy"
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538145    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy"
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538152    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy"
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538160    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy"
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538220    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy"
Jul 28 18:55:07.538227 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538228    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2fe960b-631b-4bba-8972-3eb179db4e43" containerName="main"
Jul 28 18:55:07.538479 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.538235    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="09689620-2578-4dda-95c4-19f9da87a700" containerName="istio-proxy"
Jul 28 18:55:07.541353 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.541333    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"
Jul 28 18:55:07.543741 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.543723    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xtqk5\""
Jul 28 18:55:07.549281 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.549245    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"]
Jul 28 18:55:07.636845 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.636798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2061fddd-8756-4439-a2fc-9b4e7c69140f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn\" (UID: \"2061fddd-8756-4439-a2fc-9b4e7c69140f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"
Jul 28 18:55:07.738295 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.738255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2061fddd-8756-4439-a2fc-9b4e7c69140f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn\" (UID: \"2061fddd-8756-4439-a2fc-9b4e7c69140f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"
Jul 28 18:55:07.739010 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.738989    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2061fddd-8756-4439-a2fc-9b4e7c69140f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn\" (UID: \"2061fddd-8756-4439-a2fc-9b4e7c69140f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"
Jul 28 18:55:07.853664 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.853571    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"
Jul 28 18:55:07.995501 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.995473    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"]
Jul 28 18:55:07.997924 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:55:07.997890    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2061fddd_8756_4439_a2fc_9b4e7c69140f.slice/crio-9cee1369f0169b6befc51a775a453ba3e9b91a348b9e75b5318b8cc386bbb2ae WatchSource:0}: Error finding container 9cee1369f0169b6befc51a775a453ba3e9b91a348b9e75b5318b8cc386bbb2ae: Status 404 returned error can't find the container with id 9cee1369f0169b6befc51a775a453ba3e9b91a348b9e75b5318b8cc386bbb2ae
Jul 28 18:55:07.999826 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:07.999802    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:55:08.420927 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:08.420886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn" event={"ID":"2061fddd-8756-4439-a2fc-9b4e7c69140f","Type":"ContainerStarted","Data":"9cee1369f0169b6befc51a775a453ba3e9b91a348b9e75b5318b8cc386bbb2ae"}
Jul 28 18:55:10.433904 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:10.433848    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn" event={"ID":"2061fddd-8756-4439-a2fc-9b4e7c69140f","Type":"ContainerStarted","Data":"cee431b5416ce640318b9f290484711e62bc7ec4a6d20fb31aecaaa14df76ed5"}
Jul 28 18:55:10.450117 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:10.450063    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn" podStartSLOduration=1.383255051 podStartE2EDuration="3.45004468s" totalImagesPullingTime="2.066789629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:55:10.449132862 +0000 UTC m=+3484.594685838" watchObservedRunningTime="2026-07-28 18:55:10.45004468 +0000 UTC m=+3484.595597654"
Jul 28 18:55:11.443592 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:11.443553    2575 generic.go:356] "Generic (PLEG): container finished" podID="2061fddd-8756-4439-a2fc-9b4e7c69140f" containerID="cee431b5416ce640318b9f290484711e62bc7ec4a6d20fb31aecaaa14df76ed5" exitCode=0
Jul 28 18:55:11.444139 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:11.443646    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn" event={"ID":"2061fddd-8756-4439-a2fc-9b4e7c69140f","Type":"ContainerDied","Data":"cee431b5416ce640318b9f290484711e62bc7ec4a6d20fb31aecaaa14df76ed5"}
Jul 28 18:55:12.592712 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:12.592682    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"
Jul 28 18:55:12.687746 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:12.687710    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2061fddd-8756-4439-a2fc-9b4e7c69140f-maas-api-service-ca\") pod \"2061fddd-8756-4439-a2fc-9b4e7c69140f\" (UID: \"2061fddd-8756-4439-a2fc-9b4e7c69140f\") "
Jul 28 18:55:12.688184 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:12.688159    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2061fddd-8756-4439-a2fc-9b4e7c69140f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2061fddd-8756-4439-a2fc-9b4e7c69140f" (UID: "2061fddd-8756-4439-a2fc-9b4e7c69140f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:12.788742 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:12.788705    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2061fddd-8756-4439-a2fc-9b4e7c69140f-maas-api-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:13.455327 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:13.455295    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"
Jul 28 18:55:13.455519 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:13.455328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn" event={"ID":"2061fddd-8756-4439-a2fc-9b4e7c69140f","Type":"ContainerDied","Data":"9cee1369f0169b6befc51a775a453ba3e9b91a348b9e75b5318b8cc386bbb2ae"}
Jul 28 18:55:13.455519 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:13.455377    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cee1369f0169b6befc51a775a453ba3e9b91a348b9e75b5318b8cc386bbb2ae"
Jul 28 18:55:18.209421 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:18.209380    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"]
Jul 28 18:55:18.211947 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:18.211921    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5b063e-bp2jn"]
Jul 28 18:55:18.401270 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:18.401234    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2061fddd-8756-4439-a2fc-9b4e7c69140f" path="/var/lib/kubelet/pods/2061fddd-8756-4439-a2fc-9b4e7c69140f/volumes"
Jul 28 18:55:21.791106 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:21.791064    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:55:24.977719 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:24.977681    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"]
Jul 28 18:55:24.978236 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:24.977993    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy" containerID="cri-o://4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080" gracePeriod=30
Jul 28 18:55:27.641479 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:27.641397    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 18:55:27.641878 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:27.641472    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 18:55:30.251297 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.251271    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:55:30.363087 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363034    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e044b56b-d35a-4b02-825b-ac00076f514e-istio-podinfo\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363087 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363101    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-socket\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363358 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363126    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-envoy\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363358 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363150    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e044b56b-d35a-4b02-825b-ac00076f514e-istiod-ca-cert\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363358 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363174    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-certs\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363358 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363218    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-credential-socket\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363358 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363260    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-istio-token\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363358 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-data\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363361    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26mr7\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-kube-api-access-26mr7\") pod \"e044b56b-d35a-4b02-825b-ac00076f514e\" (UID: \"e044b56b-d35a-4b02-825b-ac00076f514e\") "
Jul 28 18:55:30.363669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363423    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:30.363669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363535    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:30.363669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363545    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e044b56b-d35a-4b02-825b-ac00076f514e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:30.363669 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363622    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:30.363977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363747    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-socket\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.363977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363768    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e044b56b-d35a-4b02-825b-ac00076f514e-istiod-ca-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.363977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363781    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-workload-certs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.363977 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.363794    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-credential-socket\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.364303 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.364275    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-data" (OuterVolumeSpecName: "istio-data") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:30.365473 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.365442    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:55:30.365473 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.365468    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e044b56b-d35a-4b02-825b-ac00076f514e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 18:55:30.365625 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.365608    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-istio-token" (OuterVolumeSpecName: "istio-token") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:30.365717 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.365703    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-kube-api-access-26mr7" (OuterVolumeSpecName: "kube-api-access-26mr7") pod "e044b56b-d35a-4b02-825b-ac00076f514e" (UID: "e044b56b-d35a-4b02-825b-ac00076f514e"). InnerVolumeSpecName "kube-api-access-26mr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:30.465011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.464973    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26mr7\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-kube-api-access-26mr7\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.465011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.465005    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e044b56b-d35a-4b02-825b-ac00076f514e-istio-podinfo\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.465011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.465017    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-envoy\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.465011 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.465025    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e044b56b-d35a-4b02-825b-ac00076f514e-istio-token\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.465286 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.465032    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e044b56b-d35a-4b02-825b-ac00076f514e-istio-data\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\""
Jul 28 18:55:30.540353 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.540256    2575 generic.go:356] "Generic (PLEG): container finished" podID="e044b56b-d35a-4b02-825b-ac00076f514e" containerID="4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080" exitCode=0
Jul 28 18:55:30.540353 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.540308    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"
Jul 28 18:55:30.540353 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.540334    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" event={"ID":"e044b56b-d35a-4b02-825b-ac00076f514e","Type":"ContainerDied","Data":"4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080"}
Jul 28 18:55:30.540584 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.540390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6" event={"ID":"e044b56b-d35a-4b02-825b-ac00076f514e","Type":"ContainerDied","Data":"23ad5482fcf8c9a1025675f2f065cf760ca487fa1738b654475bba401517727e"}
Jul 28 18:55:30.540584 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.540414    2575 scope.go:117] "RemoveContainer" containerID="4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080"
Jul 28 18:55:30.549555 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.549535    2575 scope.go:117] "RemoveContainer" containerID="974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f"
Jul 28 18:55:30.558230 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.558200    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"]
Jul 28 18:55:30.558436 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.558402    2575 scope.go:117] "RemoveContainer" containerID="4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080"
Jul 28 18:55:30.558719 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:55:30.558701    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080\": container with ID starting with 4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080 not found: ID does not exist" containerID="4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080"
Jul 28 18:55:30.558787 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.558726    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080"} err="failed to get container status \"4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080\": rpc error: code = NotFound desc = could not find container \"4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080\": container with ID starting with 4f56fa38ed5e084f4345254f4bc55497134c4cd86bc72030cda5c8e30d6b1080 not found: ID does not exist"
Jul 28 18:55:30.558787 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.558742    2575 scope.go:117] "RemoveContainer" containerID="974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f"
Jul 28 18:55:30.559104 ip-10-0-132-23 kubenswrapper[2575]: E0728 18:55:30.559082    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f\": container with ID starting with 974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f not found: ID does not exist" containerID="974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f"
Jul 28 18:55:30.559166 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.559111    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f"} err="failed to get container status \"974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f\": rpc error: code = NotFound desc = could not find container \"974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f\": container with ID starting with 974077f85191ae4796431fcb82964741c0e051407afc387f93e097d16c109e6f not found: ID does not exist"
Jul 28 18:55:30.564926 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:30.564895    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-5b063e-openshift-default-857589dbb6-9qfr6"]
Jul 28 18:55:32.400463 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:32.400422    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" path="/var/lib/kubelet/pods/e044b56b-d35a-4b02-825b-ac00076f514e/volumes"
Jul 28 18:55:39.293486 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:39.293433    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:55:56.298266 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:55:56.298222    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-p7jzr"]
Jul 28 18:56:06.583246 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:06.583212    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7qx6d_fd0ad590-6443-4893-8bf9-6f7be58ecad8/manager/0.log"
Jul 28 18:56:06.712685 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:06.712637    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b5jmj_fb7e6d94-5676-4c6f-b149-6173d4d563a1/manager/0.log"
Jul 28 18:56:06.971025 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:06.970942    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7j5jt_9f9bb11c-9a86-47f7-81e5-ae63ec81dc38/server/0.log"
Jul 28 18:56:07.096401 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:07.096364    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-t5252_508100a5-0b7d-41a5-b761-2b4d53f14d9e/manager/0.log"
Jul 28 18:56:07.207721 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:07.207694    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9_c6a199ef-cf2c-4bf4-876e-3936961e980c/copy-manifests/0.log"
Jul 28 18:56:07.215380 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:07.215350    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9_c6a199ef-cf2c-4bf4-876e-3936961e980c/manager/0.log"
Jul 28 18:56:08.330225 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:08.330195    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-rskz9_82317128-bc81-4a3c-a4f1-9426f3420643/postgres/0.log"
Jul 28 18:56:09.913225 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:09.913195    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m4cbd_fcdfba35-4f17-44e5-a487-bc50d98dd536/manager/0.log"
Jul 28 18:56:10.022759 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:10.022735    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jn96v_8f7152f1-425c-4ec5-b714-c242113bbb14/kuadrant-console-plugin/0.log"
Jul 28 18:56:10.148208 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:10.148176    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5qckb_207536a4-cbcc-4960-97c7-c9316d3a7863/registry-server/0.log"
Jul 28 18:56:10.278686 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:10.278639    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5w799_2b8047d4-daab-4e14-a51b-9f219c9dad10/manager/0.log"
Jul 28 18:56:10.387061 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:10.387033    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-p7jzr_dd689fd4-6e2e-463f-aa0f-cc52af802eb6/limitador/0.log"
Jul 28 18:56:10.888400 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:10.888370    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th_29dd8780-99bf-4514-a5e0-cc99d3dd84e0/istio-proxy/0.log"
Jul 28 18:56:11.238539 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:11.238458    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cfd4f6bf9-pttnj_87c2c23e-ef58-4857-b2f1-85e0f5d667c8/kube-auth-proxy/0.log"
Jul 28 18:56:11.368787 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:11.368747    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jbnxs_72b5e147-220c-4ff1-a739-bb9bf33d95b9/istio-proxy/1.log"
Jul 28 18:56:11.746436 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:11.746405    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75c4fdbf55-r9lmg_c1e7e625-fd4c-4c3a-8c40-798c157f5173/router/0.log"
Jul 28 18:56:12.203152 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:12.203050    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-fhtrr_3675f496-9aae-46b1-b3f9-5b4233ba7eb4/main/0.log"
Jul 28 18:56:12.317778 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:12.317748    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rfrsv_670efaf0-f405-4546-981a-539aa78dc5b0/main/0.log"
Jul 28 18:56:12.542164 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:12.542131    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8s8sn_fb9ffaec-fd0d-4355-9d11-de9cedea10f5/main/0.log"
Jul 28 18:56:12.661716 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:12.661684    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-nzxnp_29a68bcf-3064-4efc-87bc-36326283be16/main/0.log"
Jul 28 18:56:20.951623 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:20.951588    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vdwk2_55e27a0d-949a-44da-9368-a62947420498/global-pull-secret-syncer/0.log"
Jul 28 18:56:21.095751 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:21.095718    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ltcl9_c27ba9c5-86c1-4f79-a3d7-425d5c2d750f/konnectivity-agent/0.log"
Jul 28 18:56:21.164426 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:21.164395    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-23.ec2.internal_75df323d7741942773952d1b2e2ae384/haproxy/0.log"
Jul 28 18:56:27.229181 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:27.229138    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m4cbd_fcdfba35-4f17-44e5-a487-bc50d98dd536/manager/0.log"
Jul 28 18:56:27.266804 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:27.266772    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jn96v_8f7152f1-425c-4ec5-b714-c242113bbb14/kuadrant-console-plugin/0.log"
Jul 28 18:56:27.331324 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:27.331284    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5qckb_207536a4-cbcc-4960-97c7-c9316d3a7863/registry-server/0.log"
Jul 28 18:56:27.596453 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:27.596407    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5w799_2b8047d4-daab-4e14-a51b-9f219c9dad10/manager/0.log"
Jul 28 18:56:27.664779 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:27.664743    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-p7jzr_dd689fd4-6e2e-463f-aa0f-cc52af802eb6/limitador/0.log"
Jul 28 18:56:29.865627 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:29.865588    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b872x_87800744-9a53-4f1f-b0ea-4a220337d4ef/node-exporter/0.log"
Jul 28 18:56:29.887966 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:29.887931    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b872x_87800744-9a53-4f1f-b0ea-4a220337d4ef/kube-rbac-proxy/0.log"
Jul 28 18:56:29.917760 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:29.917732    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b872x_87800744-9a53-4f1f-b0ea-4a220337d4ef/init-textfile/0.log"
Jul 28 18:56:31.412971 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.412928    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"]
Jul 28 18:56:31.413882 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.413835    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2061fddd-8756-4439-a2fc-9b4e7c69140f" containerName="revoke-keys"
Jul 28 18:56:31.413882 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.413879    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2061fddd-8756-4439-a2fc-9b4e7c69140f" containerName="revoke-keys"
Jul 28 18:56:31.414067 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.413927    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy"
Jul 28 18:56:31.414067 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.413938    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy"
Jul 28 18:56:31.414067 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.413953    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy"
Jul 28 18:56:31.414067 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.413962    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy"
Jul 28 18:56:31.414253 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.414149    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2061fddd-8756-4439-a2fc-9b4e7c69140f" containerName="revoke-keys"
Jul 28 18:56:31.414253 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.414163    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e044b56b-d35a-4b02-825b-ac00076f514e" containerName="istio-proxy"
Jul 28 18:56:31.418087 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.418051    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.421204 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.421171    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xtqp2\"/\"kube-root-ca.crt\""
Jul 28 18:56:31.422189 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.422155    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xtqp2\"/\"default-dockercfg-rdvgc\""
Jul 28 18:56:31.422336 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.422196    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xtqp2\"/\"openshift-service-ca.crt\""
Jul 28 18:56:31.429096 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.429066    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"]
Jul 28 18:56:31.532992 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.532950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-lib-modules\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.532992 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.532996    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-proc\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.533235 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.533105    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-podres\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.533235 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.533138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xzw8\" (UniqueName: \"kubernetes.io/projected/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-kube-api-access-6xzw8\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.533308 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.533252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-sys\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.634797 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.634753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-podres\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635039 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.634805    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xzw8\" (UniqueName: \"kubernetes.io/projected/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-kube-api-access-6xzw8\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635039 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.634842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-sys\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635039 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.634936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-lib-modules\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635039 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.634959    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-proc\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635039 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.634957    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-podres\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635039 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.635021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-sys\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635278 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.635076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-lib-modules\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.635278 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.635083    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-proc\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.643080 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.643051    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xzw8\" (UniqueName: \"kubernetes.io/projected/3df4c7fd-4978-41d5-87bf-c0f1d1f9f647-kube-api-access-6xzw8\") pod \"perf-node-gather-daemonset-qt4dk\" (UID: \"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.730560 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.730458    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:31.879705 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.879671    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"]
Jul 28 18:56:31.882758 ip-10-0-132-23 kubenswrapper[2575]: W0728 18:56:31.882728    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3df4c7fd_4978_41d5_87bf_c0f1d1f9f647.slice/crio-b142b52345b70acf134cdd3b500c747d15700193ded657d9691afec4ca5694dd WatchSource:0}: Error finding container b142b52345b70acf134cdd3b500c747d15700193ded657d9691afec4ca5694dd: Status 404 returned error can't find the container with id b142b52345b70acf134cdd3b500c747d15700193ded657d9691afec4ca5694dd
Jul 28 18:56:31.944386 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:31.944350    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-pldz2_72a6a8a7-dc6e-4b60-b06f-6f6c3497307d/networking-console-plugin/0.log"
Jul 28 18:56:32.454304 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:32.454271    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/1.log"
Jul 28 18:56:32.464813 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:32.464773    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fbhfk_4b7dc8b4-ebdd-47ae-9ef9-77a89f7ba0ef/console-operator/2.log"
Jul 28 18:56:32.853379 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:32.853341    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk" event={"ID":"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647","Type":"ContainerStarted","Data":"ea91322bd3b436e194e2de485e7fced9dba447f88e5fd7a4bcb44342e3c6a4b3"}
Jul 28 18:56:32.853571 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:32.853397    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk" event={"ID":"3df4c7fd-4978-41d5-87bf-c0f1d1f9f647","Type":"ContainerStarted","Data":"b142b52345b70acf134cdd3b500c747d15700193ded657d9691afec4ca5694dd"}
Jul 28 18:56:32.853571 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:32.853419    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:32.869303 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:32.869249    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk" podStartSLOduration=1.869233932 podStartE2EDuration="1.869233932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:56:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:56:32.868416545 +0000 UTC m=+3567.013969520" watchObservedRunningTime="2026-07-28 18:56:32.869233932 +0000 UTC m=+3567.014786981"
Jul 28 18:56:34.795302 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:34.795264    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jd452_976d2309-867a-4a03-a1c1-fc52341a4c05/dns/0.log"
Jul 28 18:56:34.834181 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:34.834149    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jd452_976d2309-867a-4a03-a1c1-fc52341a4c05/kube-rbac-proxy/0.log"
Jul 28 18:56:34.958788 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:34.958755    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqss8_599dce4a-55e5-489e-bd51-5c4251e9c3ed/dns-node-resolver/0.log"
Jul 28 18:56:35.537321 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:35.537286    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvwm8_e1a7dab8-512b-4b50-bad4-6978e3e99045/node-ca/0.log"
Jul 28 18:56:36.616101 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:36.616074    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th_29dd8780-99bf-4514-a5e0-cc99d3dd84e0/istio-proxy/0.log"
Jul 28 18:56:36.814748 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:36.814717    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cfd4f6bf9-pttnj_87c2c23e-ef58-4857-b2f1-85e0f5d667c8/kube-auth-proxy/0.log"
Jul 28 18:56:36.846930 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:36.846902    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jbnxs_72b5e147-220c-4ff1-a739-bb9bf33d95b9/istio-proxy/0.log"
Jul 28 18:56:36.903723 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:36.903627    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jbnxs_72b5e147-220c-4ff1-a739-bb9bf33d95b9/istio-proxy/1.log"
Jul 28 18:56:36.994101 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:36.994061    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75c4fdbf55-r9lmg_c1e7e625-fd4c-4c3a-8c40-798c157f5173/router/0.log"
Jul 28 18:56:37.877509 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:37.877470    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tsm5b_4d687f14-4aab-4083-b44d-1ed34c9a051c/serve-healthcheck-canary/0.log"
Jul 28 18:56:38.495960 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:38.495932    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-79fdg_4f4bb9ff-f8d8-445e-b551-eb10ad499789/kube-rbac-proxy/0.log"
Jul 28 18:56:38.518708 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:38.518679    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-79fdg_4f4bb9ff-f8d8-445e-b551-eb10ad499789/exporter/0.log"
Jul 28 18:56:38.542401 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:38.542371    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-79fdg_4f4bb9ff-f8d8-445e-b551-eb10ad499789/extractor/0.log"
Jul 28 18:56:38.869084 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:38.869007    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-qt4dk"
Jul 28 18:56:41.078525 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:41.078431    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7qx6d_fd0ad590-6443-4893-8bf9-6f7be58ecad8/manager/0.log"
Jul 28 18:56:41.500794 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:41.500755    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b5jmj_fb7e6d94-5676-4c6f-b149-6173d4d563a1/manager/0.log"
Jul 28 18:56:41.669240 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:41.669210    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7j5jt_9f9bb11c-9a86-47f7-81e5-ae63ec81dc38/server/0.log"
Jul 28 18:56:41.787260 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:41.787175    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-t5252_508100a5-0b7d-41a5-b761-2b4d53f14d9e/manager/0.log"
Jul 28 18:56:41.813104 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:41.813071    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9_c6a199ef-cf2c-4bf4-876e-3936961e980c/manager/0.log"
Jul 28 18:56:41.837142 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:41.837114    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-7pxn9_c6a199ef-cf2c-4bf4-876e-3936961e980c/copy-manifests/0.log"
Jul 28 18:56:42.365182 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:42.365154    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th_29dd8780-99bf-4514-a5e0-cc99d3dd84e0/istio-proxy/0.log"
Jul 28 18:56:42.537440 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:42.537405    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cfd4f6bf9-pttnj_87c2c23e-ef58-4857-b2f1-85e0f5d667c8/kube-auth-proxy/0.log"
Jul 28 18:56:42.577516 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:42.570205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jbnxs_72b5e147-220c-4ff1-a739-bb9bf33d95b9/istio-proxy/0.log"
Jul 28 18:56:42.620478 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:42.620381    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jbnxs_72b5e147-220c-4ff1-a739-bb9bf33d95b9/istio-proxy/1.log"
Jul 28 18:56:42.737333 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:42.737299    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75c4fdbf55-r9lmg_c1e7e625-fd4c-4c3a-8c40-798c157f5173/router/0.log"
Jul 28 18:56:44.263773 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:44.263740    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vt7th_29dd8780-99bf-4514-a5e0-cc99d3dd84e0/istio-proxy/0.log"
Jul 28 18:56:44.443983 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:44.443948    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cfd4f6bf9-pttnj_87c2c23e-ef58-4857-b2f1-85e0f5d667c8/kube-auth-proxy/0.log"
Jul 28 18:56:44.468302 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:44.468254    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jbnxs_72b5e147-220c-4ff1-a739-bb9bf33d95b9/istio-proxy/0.log"
Jul 28 18:56:44.518092 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:44.517990    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jbnxs_72b5e147-220c-4ff1-a739-bb9bf33d95b9/istio-proxy/1.log"
Jul 28 18:56:44.581106 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:44.581073    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75c4fdbf55-r9lmg_c1e7e625-fd4c-4c3a-8c40-798c157f5173/router/0.log"
Jul 28 18:56:48.703488 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:48.703448    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc_f0fd56f1-1ed2-4ffc-b730-b575cfe7380e/kube-storage-version-migrator-operator/1.log"
Jul 28 18:56:48.705087 ip-10-0-132-23 kubenswrapper[2575]: I0728 18:56:48.705060    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-s5thc_f0fd56f1-1ed2-4ffc-b730-b575cfe7380e/kube-storage-version-migrator-operator/0.log"