Jul 27 17:18:44.031716 ip-10-0-131-8 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 27 17:18:44.031727 ip-10-0-131-8 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 27 17:18:44.031735 ip-10-0-131-8 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 27 17:18:44.031969 ip-10-0-131-8 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 27 17:18:53.483696 ip-10-0-131-8 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 27 17:18:53.483712 ip-10-0-131-8 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f2df667125e64efa97f6851d626615d7 --
Jul 27 17:20:42.623570 ip-10-0-131-8 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 17:20:43.032004 ip-10-0-131-8 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:20:43.032004 ip-10-0-131-8 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 17:20:43.032004 ip-10-0-131-8 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:20:43.032004 ip-10-0-131-8 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 17:20:43.032004 ip-10-0-131-8 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:20:43.032958 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.032855    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 17:20:43.036379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036364    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:20:43.036379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036378    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036382    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036385    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036388    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036391    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036394    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036397    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036399    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036402    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036404    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036407    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036410    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036412    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036419    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036422    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036425    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036428    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036430    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036433    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036435    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:20:43.036442 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036437    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036440    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036442    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036445    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036448    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036451    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036453    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036456    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036459    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036461    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036463    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036466    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036469    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036471    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036473    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036476    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036478    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036480    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036483    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036486    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:20:43.036919 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036488    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036491    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036493    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036495    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036498    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036500    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036502    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036505    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036507    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036509    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036513    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036517    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036520    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036523    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036527    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036530    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036533    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036535    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036538    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036541    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:20:43.037412 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036543    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036546    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036549    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036551    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036554    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036558    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036561    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036564    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036567    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036570    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036574    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036576    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036578    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036581    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036583    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036586    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036588    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036592    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036595    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:20:43.037900 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036598    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036601    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036603    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036606    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036609    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036611    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036614    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036617    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036620    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036622    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036625    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036627    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036631    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036633    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036635    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036638    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036640    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036643    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.036645    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037343    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:20:43.038404 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037350    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037353    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037356    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037359    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037362    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037365    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037368    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037370    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037373    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037376    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037378    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037382    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037386    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037390    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037393    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037396    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037398    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037402    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037405    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:20:43.038869 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037408    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037411    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037415    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037417    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037421    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037423    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037426    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037429    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037432    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037435    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037437    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037440    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037442    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037445    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037447    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037450    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037453    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037455    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037458    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037461    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:20:43.039336 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037463    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037466    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037469    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037472    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037474    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037477    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037480    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037482    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037485    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037488    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037490    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037493    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037496    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037498    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037501    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037506    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037509    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037511    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037515    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:20:43.039802 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037517    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037520    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037523    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037526    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037528    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037531    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037534    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037536    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037539    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037542    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037544    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037547    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037550    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037552    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037555    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037558    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037561    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037564    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037567    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037569    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:20:43.040237 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037572    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037574    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037577    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037579    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037583    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037586    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037589    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037592    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037594    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037597    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037600    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037603    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037605    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037608    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037610    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037613    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037616    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037618    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037621    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037624    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:20:43.040728 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037965    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037970    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037973    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037976    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037979    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037982    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037984    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037987    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037990    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037993    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037995    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.037998    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038000    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038003    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038006    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038008    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038024    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038028    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038031    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038033    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:20:43.041238 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038036    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038039    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038041    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038044    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038047    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038050    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038052    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038068    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038071    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038073    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038075    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038078    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038080    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038083    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038085    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038088    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038090    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038093    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038096    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:20:43.041745 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038100    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038104    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038106    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038109    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038112    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038116    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038119    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038122    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038125    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038127    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038130    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038132    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038135    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038137    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038140    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038142    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038144    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038147    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038149    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:20:43.042226 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038152    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038154    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038157    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038159    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038162    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038165    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038167    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038170    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038172    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038174    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038176    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038179    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038182    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038184    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038187    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038189    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038191    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038193    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038196    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038198    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:20:43.042682 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038201    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038203    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038205    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038208    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038211    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038213    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038217    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038220    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038224    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038226    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038229    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038231    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038234    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038236    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038239    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038241    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038243    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038247    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038249    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038251    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:20:43.043146 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038254    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038332    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038339    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038345    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038349    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038353    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038356    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038360    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038364    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038367    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038370    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038373    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038376    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038379    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038382    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038385    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038388    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038391    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038394    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038398    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038401    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038404    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038407    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038411    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 17:20:43.043663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038414    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038417    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038420    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038423    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038427    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038429    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038432    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038435    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038438    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038442    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038445    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038447    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038450    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038453    2565 flags.go:64] FLAG: --enable-server="true"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038461    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038465    2565 flags.go:64] FLAG: --event-burst="100"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038468    2565 flags.go:64] FLAG: --event-qps="50"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038471    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038474    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038477    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038480    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038483    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038486    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038491    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038494    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 17:20:43.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038497    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038500    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038502    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038507    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038510    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038512    2565 flags.go:64] FLAG: --feature-gates=""
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038516    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038519    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038522    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038525    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038527    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038530    2565 flags.go:64] FLAG: --help="false"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038533    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038536    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038539    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038541    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038544    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038548    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038550    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038553    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038556    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038559    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038562    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038565    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 17:20:43.044898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038567    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038570    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038573    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038576    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038578    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038581    2565 flags.go:64] FLAG: --lock-file=""
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038585    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038588    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038591    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038596    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038599    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038602    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038605    2565 flags.go:64] FLAG: --logging-format="text"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038608    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038611    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038614    2565 flags.go:64] FLAG: --manifest-url=""
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038617    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038620    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038623    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038627    2565 flags.go:64] FLAG: --max-pods="110"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038630    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038633    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038636    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038638    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038641    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 17:20:43.045498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038644    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038647    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038654    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038657    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038660    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038663    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038666    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038671    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038674    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038677    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038680    2565 flags.go:64] FLAG: --port="10250"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038682    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038685    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-055e740cc00c0b1f6"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038688    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038692    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038695    2565 flags.go:64] FLAG: --register-node="true"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038698    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038701    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038705    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038708    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038710    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038714    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038717    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038720    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038722    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 17:20:43.046144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038725    2565 flags.go:64] FLAG: --runonce="false"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038728    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038731    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038734    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038737    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038740    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038743    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038746    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038749    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038751    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038754    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038757    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038760    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038762    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038766    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038768    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038773    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038776    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038779    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038783    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038785    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038790    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038792    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038795    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038798    2565 flags.go:64] FLAG: --v="2"
Jul 27 17:20:43.046785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038803    2565 flags.go:64] FLAG: --version="false"
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038807    2565 flags.go:64] FLAG: --vmodule=""
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038811    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.038815    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038916    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038920    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038923    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038926    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038928    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038931    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038933    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038936    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038938    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038941    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038943    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038945    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038948    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038950    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038953    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038955    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:20:43.047379 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038958    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038960    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038963    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038966    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038968    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038971    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038973    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038976    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038979    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038982    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038984    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038987    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038991    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038993    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038996    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.038999    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039001    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039004    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039006    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039009    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:20:43.047875 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039012    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039014    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039017    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039019    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039021    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039024    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039027    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039029    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039031    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039034    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039036    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039039    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039041    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039044    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039046    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039049    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039052    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039054    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039057    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039059    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:20:43.048386 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039063    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039065    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039068    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039070    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039074    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039077    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039079    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039082    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039084    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039087    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039089    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039092    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039094    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039097    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039099    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039102    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039105    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039107    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039109    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039112    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:20:43.048854 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039114    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039117    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039119    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039122    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039124    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039127    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039129    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039132    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039134    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039137    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039140    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039144    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039148    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039150    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039153    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039157    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039162    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039165    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039167    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039170    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:20:43.049636 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039173    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039176    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.039180    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.039785    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.047325    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.047340    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.047354    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.047360    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047422    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047426    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047429    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047432    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047435    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047438    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:20:43.050283 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047441    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047443    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047446    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047448    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047451    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047454    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047456    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047459    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047462    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047466    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047469    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047472    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047475    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047477    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047480    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047483    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047485    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047488    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047490    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:20:43.050712 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047493    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047495    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047497    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047500    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047509    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047512    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047515    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047518    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047520    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047523    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047526    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047528    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047530    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047533    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047535    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047538    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047540    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047542    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047545    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047547    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:20:43.051152 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047550    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047552    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047555    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047557    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047559    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047562    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047564    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047567    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047570    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047572    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047575    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047577    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047579    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047582    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047584    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047587    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047589    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047597    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047600    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047602    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:20:43.051630 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047605    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047607    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047610    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047612    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047614    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047616    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047619    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047621    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047623    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047626    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047628    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047630    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047633    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047635    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047638    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047641    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047645    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047647    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047650    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047653    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:20:43.052083 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047655    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047657    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047660    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047662    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047664    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047667    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047669    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047671    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047674    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047676    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047684    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047686    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047689    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047691    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.047696    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:20:43.052602 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047840    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047845    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047848    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047852    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047855    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047857    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047860    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047863    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047865    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047868    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047870    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047873    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047875    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047878    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047880    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047883    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047886    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047889    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047891    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047894    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:20:43.052992 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047896    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047899    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047901    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047903    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047906    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047908    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047910    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047913    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047921    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047923    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047926    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047928    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047930    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047933    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047935    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047937    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047940    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047943    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047946    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:20:43.053485 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047949    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047952    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047954    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047956    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047959    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047961    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047964    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047967    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047969    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047972    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047974    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047977    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047979    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047981    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047984    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047986    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047989    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047991    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047993    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.047995    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:20:43.053946 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048004    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048006    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048015    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048017    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048020    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048022    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048025    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048027    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048029    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048032    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048034    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048036    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048039    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048041    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048043    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048047    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048050    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048053    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048056    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:20:43.054500 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048061    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048064    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048067    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048069    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048072    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048075    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048077    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048080    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048082    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048085    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048087    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048089    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048092    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048094    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048097    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048100    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048108    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048111    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048113    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048115    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:20:43.054935 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:43.048118    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:20:43.055407 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.048122    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:20:43.055407 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.048217    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 17:20:43.055407 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.053127    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 17:20:43.055407 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.054404    2565 server.go:1019] "Starting client certificate rotation"
Jul 27 17:20:43.055407 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.054499    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:20:43.055407 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.054536    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:20:43.056913 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.056899    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:20:43.058351 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.058288    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:20:43.064826 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.064809    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 27 17:20:43.067047 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.067028    2565 log.go:25] "Validated CRI v1 image API"
Jul 27 17:20:43.069032 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.069017    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 17:20:43.071439 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.071417    2565 fs.go:135] Filesystem UUIDs: map[792994a2-18fc-4af1-9b63-befdf73356a7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bb03b292-42ba-472b-b56c-93510d461bea:/dev/nvme0n1p4]
Jul 27 17:20:43.071503 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.071433    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 17:20:43.077119 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.077005    2565 manager.go:216] Machine: {Timestamp:2026-07-27 17:20:43.075601146 +0000 UTC m=+0.336576335 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3198287 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20f438a761db5a858f56ec3bfc6a07 SystemUUID:ec20f438-a761-db5a-858f-56ec3bfc6a07 BootID:f2df6671-25e6-4efa-97f6-851d626615d7 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ac:7d:77:a4:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ac:7d:77:a4:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:86:a9:da:fe:4c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 17:20:43.077119 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.077108    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 17:20:43.077269 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.077184    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 17:20:43.080141 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.080117    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 17:20:43.080257 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.080141    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-8.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 17:20:43.081344 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.081333    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 17:20:43.081377 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.081347    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 17:20:43.081377 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.081354    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:20:43.082082 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.082072    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:20:43.082970 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.082961    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 17:20:43.086192 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.086179    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:20:43.086316 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.086289    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 17:20:43.086991 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.086979    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 17:20:43.087033 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.086998    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 17:20:43.087033 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.087013    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 17:20:43.087033 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.087025    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 27 17:20:43.087152 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.087036    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 17:20:43.087762 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.087749    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:20:43.087815 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.087771    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:20:43.088548 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.088530    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:20:43.090752 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.090726    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 17:20:43.092619 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.092599    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 17:20:43.092697 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.092629    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 17:20:43.094182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094169    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094185    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094193    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094199    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094207    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094212    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094220    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094225    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 17:20:43.094233 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094232    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 17:20:43.094459 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094237    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 17:20:43.094459 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094246    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 17:20:43.094459 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.094251    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 17:20:43.095103 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.095094    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 17:20:43.095130 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.095104    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 17:20:43.098042 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.098027    2565 server.go:1268] "Started kubelet"
Jul 27 17:20:43.098136 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.098112    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 17:20:43.098191 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.098130    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 17:20:43.098250 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.098198    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 17:20:43.098798 ip-10-0-131-8 systemd[1]: Started Kubernetes Kubelet.
Jul 27 17:20:43.099779 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.099766    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 17:20:43.102416 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.102399    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 17:20:43.102899 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.102882    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 17:20:43.103788 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.103768    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 17:20:43.103979 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.103968    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-8.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 17:20:43.104030 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.104011    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 17:20:43.104750 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.103752    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-8.ec2.internal.18c6353e1787d09f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-8.ec2.internal,UID:ip-10-0-131-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-8.ec2.internal,},FirstTimestamp:2026-07-27 17:20:43.098001567 +0000 UTC m=+0.358976758,LastTimestamp:2026-07-27 17:20:43.098001567 +0000 UTC m=+0.358976758,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-8.ec2.internal,}"
Jul 27 17:20:43.108165 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108146    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 17:20:43.108165 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108155    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 17:20:43.108378 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.108357    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.108689 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108675    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 17:20:43.108771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108694    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 17:20:43.108845 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108710    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 17:20:43.108845 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108802    2565 factory.go:55] Registering systemd factory
Jul 27 17:20:43.108977 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108856    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 27 17:20:43.108977 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108964    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 17:20:43.108977 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.108975    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 17:20:43.109209 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.109191    2565 factory.go:153] Registering CRI-O factory
Jul 27 17:20:43.109209 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.109212    2565 factory.go:223] Registration of the crio container factory successfully
Jul 27 17:20:43.109344 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.109259    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 17:20:43.109344 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.109281    2565 factory.go:103] Registering Raw factory
Jul 27 17:20:43.109344 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.109316    2565 manager.go:1196] Started watching for new ooms in manager
Jul 27 17:20:43.110654 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.110639    2565 manager.go:349] Starting recovery of all containers
Jul 27 17:20:43.112491 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.112461    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 17:20:43.112491 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.112480    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 17:20:43.121427 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.121404    2565 manager.go:354] Recovery completed
Jul 27 17:20:43.123272 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.123202    2565 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 27 17:20:43.126121 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.126109    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:43.127940 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.127920    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:43.128008 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.127958    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:43.128008 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.127974    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:43.128488 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.128473    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 17:20:43.128543 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.128489    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 17:20:43.128543 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.128532    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:20:43.128543 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.128534    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:43.129181 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.129168    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:43.129247 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.129193    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:43.129247 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.129203    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:43.130952 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.130890    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-8.ec2.internal.18c6353e1950a6a6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-8.ec2.internal,UID:ip-10-0-131-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-8.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-8.ec2.internal,},FirstTimestamp:2026-07-27 17:20:43.127940774 +0000 UTC m=+0.388915970,LastTimestamp:2026-07-27 17:20:43.127940774 +0000 UTC m=+0.388915970,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-8.ec2.internal,}"
Jul 27 17:20:43.131190 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.131177    2565 policy_none.go:49] "None policy: Start"
Jul 27 17:20:43.131234 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.131196    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 17:20:43.131234 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.131209    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 17:20:43.133349 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.133338    2565 policy_none.go:47] "Start"
Jul 27 17:20:43.139986 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.139930    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-8.ec2.internal.18c6353e1951088a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-8.ec2.internal,UID:ip-10-0-131-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-8.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-8.ec2.internal,},FirstTimestamp:2026-07-27 17:20:43.127965834 +0000 UTC m=+0.388941027,LastTimestamp:2026-07-27 17:20:43.127965834 +0000 UTC m=+0.388941027,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-8.ec2.internal,}"
Jul 27 17:20:43.143976 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.143962    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nddtl"
Jul 27 17:20:43.150527 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.150509    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nddtl"
Jul 27 17:20:43.152722 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.152558    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-8.ec2.internal.18c6353e195140db  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-8.ec2.internal,UID:ip-10-0-131-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-8.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-8.ec2.internal,},FirstTimestamp:2026-07-27 17:20:43.127980251 +0000 UTC m=+0.388955440,LastTimestamp:2026-07-27 17:20:43.127980251 +0000 UTC m=+0.388955440,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-8.ec2.internal,}"
Jul 27 17:20:43.167811 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.167796    2565 manager.go:337] "Starting Device Plugin manager"
Jul 27 17:20:43.167898 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.167839    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 17:20:43.167898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.167852    2565 server.go:87] "Starting device plugin registration server"
Jul 27 17:20:43.168070 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.168058    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 17:20:43.168125 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.168071    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 17:20:43.168182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.168166    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 17:20:43.168262 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.168251    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 17:20:43.168327 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.168263    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 17:20:43.168674 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.168652    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 17:20:43.168763 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.168701    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.248260 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.248229    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 17:20:43.249418 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.249395    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 17:20:43.249496 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.249431    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 17:20:43.249496 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.249450    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 17:20:43.249496 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.249484    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 17:20:43.251596 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.251576    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:20:43.268311 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.268283    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:43.269064 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.269045    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:43.269115 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.269074    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:43.269115 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.269085    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:43.269115 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.269107    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.278286 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.278272    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.278373 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.278292    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-8.ec2.internal\": node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.310092 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.310046    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.349682 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.349658    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal"]
Jul 27 17:20:43.349792 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.349737    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:43.350544 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.350531    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:43.350612 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.350556    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:43.350612 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.350571    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:43.352823 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.352812    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:43.352943 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.352930    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.352989 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.352959    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:43.353437 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.353421    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:43.353491 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.353425    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:43.353491 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.353467    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:43.353491 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.353479    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:43.353589 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.353447    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:43.353589 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.353522    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:43.355588 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.355575    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.355659 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.355597    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:43.356193 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.356176    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:43.356265 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.356207    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:43.356265 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.356222    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:43.370890 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.370867    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-8.ec2.internal\" not found" node="ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.374966 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.374948    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-8.ec2.internal\" not found" node="ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.410354 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.410334    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.410430 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.410377    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fdacb45223ffcd32a88e9773f54ccf6a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal\" (UID: \"fdacb45223ffcd32a88e9773f54ccf6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.410430 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.410397    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdacb45223ffcd32a88e9773f54ccf6a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal\" (UID: \"fdacb45223ffcd32a88e9773f54ccf6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.410430 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.410412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a952e817f62df0bda7ddea87b49c2739-config\") pod \"kube-apiserver-proxy-ip-10-0-131-8.ec2.internal\" (UID: \"a952e817f62df0bda7ddea87b49c2739\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.510512 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.510476    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.510616 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.510525    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a952e817f62df0bda7ddea87b49c2739-config\") pod \"kube-apiserver-proxy-ip-10-0-131-8.ec2.internal\" (UID: \"a952e817f62df0bda7ddea87b49c2739\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.510616 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.510554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fdacb45223ffcd32a88e9773f54ccf6a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal\" (UID: \"fdacb45223ffcd32a88e9773f54ccf6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.510616 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.510578    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdacb45223ffcd32a88e9773f54ccf6a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal\" (UID: \"fdacb45223ffcd32a88e9773f54ccf6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.510727 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.510612    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a952e817f62df0bda7ddea87b49c2739-config\") pod \"kube-apiserver-proxy-ip-10-0-131-8.ec2.internal\" (UID: \"a952e817f62df0bda7ddea87b49c2739\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.510727 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.510626    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fdacb45223ffcd32a88e9773f54ccf6a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal\" (UID: \"fdacb45223ffcd32a88e9773f54ccf6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.510727 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.510670    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdacb45223ffcd32a88e9773f54ccf6a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal\" (UID: \"fdacb45223ffcd32a88e9773f54ccf6a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.611190 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.611133    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.674250 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.674227    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.677865 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:43.677850    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:43.711212 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.711183    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.811758 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.811723    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:43.912737 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:43.912671    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.013789 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:44.013758    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.054015 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.053993    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 17:20:44.054426 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.054129    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 17:20:44.114182 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:44.114159    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.155340 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.155307    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 17:15:43 +0000 UTC" deadline="2028-01-29 13:59:51.516481494 +0000 UTC"
Jul 27 17:20:44.155340 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.155337    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13220h39m7.361148502s"
Jul 27 17:20:44.214594 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:44.214547    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.256944 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.256926    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:20:44.298252 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.298228    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:20:44.299102 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.299086    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:20:44.299171 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.299114    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:20:44.299171 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.299124    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:20:44.314605 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:44.314584    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.415013 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:44.414988    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.515430 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:44.515352    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.616023 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:44.615992    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-8.ec2.internal\" not found"
Jul 27 17:20:44.665696 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.665673    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:20:44.689871 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.689841    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:20:44.709174 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.709148    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:44.722474 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.722454    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:20:44.723464 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.723451    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal"
Jul 27 17:20:44.732608 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.732594    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:20:44.755109 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:44.755078    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda952e817f62df0bda7ddea87b49c2739.slice/crio-398996fa2ac18f863abe05c3b053ca7877e51714dd30b2764dfabbe0ad61c460 WatchSource:0}: Error finding container 398996fa2ac18f863abe05c3b053ca7877e51714dd30b2764dfabbe0ad61c460: Status 404 returned error can't find the container with id 398996fa2ac18f863abe05c3b053ca7877e51714dd30b2764dfabbe0ad61c460
Jul 27 17:20:44.755545 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:44.755526    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdacb45223ffcd32a88e9773f54ccf6a.slice/crio-75b3c39774ed80dbeff1c2c1cdaec8ef0e21c3be95705ff6b175b39b72cd5591 WatchSource:0}: Error finding container 75b3c39774ed80dbeff1c2c1cdaec8ef0e21c3be95705ff6b175b39b72cd5591: Status 404 returned error can't find the container with id 75b3c39774ed80dbeff1c2c1cdaec8ef0e21c3be95705ff6b175b39b72cd5591
Jul 27 17:20:44.760272 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:44.760259    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:20:45.082324 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.082284    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:20:45.087541 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.087530    2565 apiserver.go:52] "Watching apiserver"
Jul 27 17:20:45.094839 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.094820    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 17:20:45.095200 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.095179    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb","openshift-multus/multus-525fp","openshift-network-diagnostics/network-check-target-bdtq8","openshift-network-operator/iptables-alerter-bvxjr","openshift-ovn-kubernetes/ovnkube-node-hv7q4","openshift-cluster-node-tuning-operator/tuned-czzk6","openshift-dns/node-resolver-2kb9q","openshift-image-registry/node-ca-c7rx4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal","openshift-multus/multus-additional-cni-plugins-nfjn2","openshift-multus/network-metrics-daemon-dls6k","kube-system/global-pull-secret-syncer-l288b","kube-system/konnectivity-agent-vhdzk"]
Jul 27 17:20:45.099001 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.098978    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.101589 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.101574    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.101661 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.101627    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:45.102176 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.102162    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xp4lh\""
Jul 27 17:20:45.102234 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.102211    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 17:20:45.103057 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.103039    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:20:45.103647 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.103632    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.105690 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.105670    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:45.105764 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.105724    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:45.106242 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.106228    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 17:20:45.106304 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.106237    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 17:20:45.106465 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.106454    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wpb8l\""
Jul 27 17:20:45.106676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.106661    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 17:20:45.106717 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.106661    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 17:20:45.107758 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.107741    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.108924 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.108908    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 17:20:45.110050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.110031    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.110668 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.110647    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:20:45.110761 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.110682    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 17:20:45.110761 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.110691    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tgcgh\""
Jul 27 17:20:45.110761 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.110717    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 17:20:45.112048 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.112033    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.112836 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.112810    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 17:20:45.113082 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.113059    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qg782\""
Jul 27 17:20:45.113141 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.113079    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 17:20:45.113141 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.113096    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 17:20:45.113251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.113075    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 17:20:45.114226 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.114212    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.114585 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.114566    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 17:20:45.114675 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.114650    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 17:20:45.114736 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.114679    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 17:20:45.114787 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.114771    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 17:20:45.114824 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.114801    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vl4wk\""
Jul 27 17:20:45.115345 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.115323    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 17:20:45.116564 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.116549    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.117279 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.117263    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9j5fv\""
Jul 27 17:20:45.117728 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.117706    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 17:20:45.118266 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.118238    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49d7\" (UniqueName: \"kubernetes.io/projected/3408bd3e-f7d9-43a3-a172-f8c55e43e578-kube-api-access-x49d7\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovnkube-config\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119444    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119458    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119623    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6chq\""
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119719    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-modprobe-d\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119855    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysctl-conf\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.119890    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmd9r\" (UniqueName: \"kubernetes.io/projected/4e6e84a1-2f73-45d6-a267-f4256d66e13f-kube-api-access-nmd9r\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.120050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120046    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-dbus\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120072    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtkg7\" (UniqueName: \"kubernetes.io/projected/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-kube-api-access-mtkg7\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-slash\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120139    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-log-socket\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120171    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-lib-modules\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120221    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120227    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120246    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120273    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-kubernetes\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-os-release\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120347    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysconfig\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120491    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3408bd3e-f7d9-43a3-a172-f8c55e43e578-cni-binary-copy\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.120580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120559    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-var-lib-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120598    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-sys\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120625    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-host\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-kubelet\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120763    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-daemon-config\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120831    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-etc-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-cni-bin\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120903    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-node-log\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-cni-netd\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.120990    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-system-cni-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-host-slash\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.121183 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121095    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-env-overrides\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-hostroot\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-etc-kubernetes\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-kubelet\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121335    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysctl-d\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e6e84a1-2f73-45d6-a267-f4256d66e13f-tmp\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121420    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-cni-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121537    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-multus-certs\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121563    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-iptables-alerter-script\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-systemd-units\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.121676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121608    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121639    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-run\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121707    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-var-lib-kubelet\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121748    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-cni-multus\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121766    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-kubelet-config\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121782    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-socket-dir-parent\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-netns\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-ovn\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121892    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztphc\" (UniqueName: \"kubernetes.io/projected/b125957e-6328-4af6-bc35-ed60ae66dc7e-kube-api-access-ztphc\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.121992    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-k8s-cni-cncf-io\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122079    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-cni-bin\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122122    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-run-netns\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122182    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-tuned\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122230    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-cnibin\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122240    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122270    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-conf-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122342    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovn-node-metrics-cert\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-systemd\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122503    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovnkube-script-lib\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.122941 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.122541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-systemd\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.123375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.123342    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 17:20:45.123453 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.123362    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 17:20:45.123995 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.123771    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pjdbj\""
Jul 27 17:20:45.124258 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.124241    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:20:45.124568 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.124556    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:45.124624 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.124610    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:20:45.125267 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.125254    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pxtmn\""
Jul 27 17:20:45.125331 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.125285    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 17:20:45.125399 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.125259    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 17:20:45.146810 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.146793    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9hgtd"
Jul 27 17:20:45.156363 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.156348    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9hgtd"
Jul 27 17:20:45.209935 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.209913    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 17:20:45.223242 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-netns\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.223327 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-ovn\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.223327 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223270    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztphc\" (UniqueName: \"kubernetes.io/projected/b125957e-6328-4af6-bc35-ed60ae66dc7e-kube-api-access-ztphc\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.223427 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223331    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-sys-fs\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.223427 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223362    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-netns\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.223427 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223368    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f91c41b-a2d4-4635-840f-09bf2302fd91-hosts-file\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.223427 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223338    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-ovn\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.223574 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223436    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tjp4\" (UniqueName: \"kubernetes.io/projected/4f91c41b-a2d4-4635-840f-09bf2302fd91-kube-api-access-8tjp4\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.223574 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223463    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-os-release\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.223574 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223513    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-k8s-cni-cncf-io\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.223574 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223544    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-cni-bin\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.223574 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-run-netns\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-run-netns\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-k8s-cni-cncf-io\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223621    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-tuned\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-cni-bin\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223643    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223689    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8pvv\" (UniqueName: \"kubernetes.io/projected/6b483d54-7e05-4952-82b2-e25106bd4549-kube-api-access-x8pvv\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223714    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-cnibin\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.223771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-conf-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovn-node-metrics-cert\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223828    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-cnibin\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-conf-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223882    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-systemd\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223906    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223930    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovnkube-script-lib\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223967    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-systemd\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.223968    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-systemd\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224004    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-run-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-host\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224100    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-serviceca\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224105    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-systemd\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224156    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x49d7\" (UniqueName: \"kubernetes.io/projected/3408bd3e-f7d9-43a3-a172-f8c55e43e578-kube-api-access-x49d7\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.224182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovnkube-config\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224211    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-modprobe-d\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224234    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysctl-conf\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmd9r\" (UniqueName: \"kubernetes.io/projected/4e6e84a1-2f73-45d6-a267-f4256d66e13f-kube-api-access-nmd9r\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224279    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-dbus\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtkg7\" (UniqueName: \"kubernetes.io/projected/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-kube-api-access-mtkg7\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224359    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-slash\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224382    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-log-socket\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-lib-modules\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224428    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-registration-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224486    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-slash\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224494    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysctl-conf\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224499    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-system-cni-dir\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224379    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-modprobe-d\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224632    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224664    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-kubernetes\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.224763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8rhk\" (UniqueName: \"kubernetes.io/projected/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-kube-api-access-t8rhk\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224703    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-dbus\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224741    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nql6\" (UniqueName: \"kubernetes.io/projected/8688d57f-b060-4a55-a4aa-a503363dfe2c-kube-api-access-7nql6\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224743    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovnkube-config\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224771    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-os-release\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.224834    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-lib-modules\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224874    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-kubernetes\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224894    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovnkube-script-lib\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224933    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224968    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224984    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-log-socket\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.224949    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-os-release\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.225038    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret podName:e4e307bf-7bb5-44dc-8dc0-890bbafffb78 nodeName:}" failed. No retries permitted until 2026-07-27 17:20:45.724996398 +0000 UTC m=+2.985971574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret") pod "global-pull-secret-syncer-l288b" (UID: "e4e307bf-7bb5-44dc-8dc0-890bbafffb78") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysconfig\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225103    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3408bd3e-f7d9-43a3-a172-f8c55e43e578-cni-binary-copy\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.225502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225112    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysconfig\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225126    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-var-lib-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225147    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-sys\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-host\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225203    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-socket-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225238    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-host\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225179    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-var-lib-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225206    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-sys\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225236    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-kubelet\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225279    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-kubelet\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225314    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-daemon-config\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225339    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-etc-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225356    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-cni-bin\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-cnibin\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225421    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-etc-openvswitch\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225429    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-node-log\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225429    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-cni-bin\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-cni-netd\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.226285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-node-log\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60f24a69-e287-4fc4-a659-e0d275e17215-konnectivity-ca\") pod \"konnectivity-agent-vhdzk\" (UID: \"60f24a69-e287-4fc4-a659-e0d275e17215\") " pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225523    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-cni-netd\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225591    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-system-cni-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225616    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-host-slash\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225637    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-env-overrides\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225661    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-hostroot\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225668    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-host-slash\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225684    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-etc-kubernetes\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225667    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3408bd3e-f7d9-43a3-a172-f8c55e43e578-cni-binary-copy\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225686    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-system-cni-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225721    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-hostroot\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225739    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-etc-kubernetes\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-kubelet\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225749    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-daemon-config\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225771    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysctl-d\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225800    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-kubelet\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e6e84a1-2f73-45d6-a267-f4256d66e13f-tmp\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60f24a69-e287-4fc4-a659-e0d275e17215-agent-certs\") pod \"konnectivity-agent-vhdzk\" (UID: \"60f24a69-e287-4fc4-a659-e0d275e17215\") " pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225873    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-cni-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225889    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-sysctl-d\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-multus-certs\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-iptables-alerter-script\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225952    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-cni-dir\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225958    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-systemd-units\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226007    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-device-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.225953    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-run-multus-certs\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226047    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b125957e-6328-4af6-bc35-ed60ae66dc7e-env-overrides\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226047    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226095    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226105    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-run\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226131    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b125957e-6328-4af6-bc35-ed60ae66dc7e-systemd-units\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226140    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-var-lib-kubelet\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.227657 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-run\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226213    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-cni-multus\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226245    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e6e84a1-2f73-45d6-a267-f4256d66e13f-var-lib-kubelet\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226283    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-host-var-lib-cni-multus\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-kubelet-config\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226356    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f91c41b-a2d4-4635-840f-09bf2302fd91-tmp-dir\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226385    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-kubelet-config\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7r7r\" (UniqueName: \"kubernetes.io/projected/12fa5ebb-f199-4903-8d8d-d571608e044b-kube-api-access-x7r7r\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226491    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-socket-dir-parent\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226493    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-iptables-alerter-script\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.226530    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3408bd3e-f7d9-43a3-a172-f8c55e43e578-multus-socket-dir-parent\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.227314    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e6e84a1-2f73-45d6-a267-f4256d66e13f-etc-tuned\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.227425    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b125957e-6328-4af6-bc35-ed60ae66dc7e-ovn-node-metrics-cert\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.228127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.227636    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e6e84a1-2f73-45d6-a267-f4256d66e13f-tmp\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.237043 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.237027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztphc\" (UniqueName: \"kubernetes.io/projected/b125957e-6328-4af6-bc35-ed60ae66dc7e-kube-api-access-ztphc\") pod \"ovnkube-node-hv7q4\" (UID: \"b125957e-6328-4af6-bc35-ed60ae66dc7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.238063 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.238042    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:20:45.238148 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.238068    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:20:45.238148 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.238081    2565 projected.go:196] Error preparing data for projected volume kube-api-access-96k4k for pod openshift-network-diagnostics/network-check-target-bdtq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:45.238148 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.238146    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k podName:c0ec967f-de07-46cc-899c-c66c49a81afd nodeName:}" failed. No retries permitted until 2026-07-27 17:20:45.738121074 +0000 UTC m=+2.999096269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-96k4k" (UniqueName: "kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k") pod "network-check-target-bdtq8" (UID: "c0ec967f-de07-46cc-899c-c66c49a81afd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:45.239134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.239110    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49d7\" (UniqueName: \"kubernetes.io/projected/3408bd3e-f7d9-43a3-a172-f8c55e43e578-kube-api-access-x49d7\") pod \"multus-525fp\" (UID: \"3408bd3e-f7d9-43a3-a172-f8c55e43e578\") " pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.239518 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.239503    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmd9r\" (UniqueName: \"kubernetes.io/projected/4e6e84a1-2f73-45d6-a267-f4256d66e13f-kube-api-access-nmd9r\") pod \"tuned-czzk6\" (UID: \"4e6e84a1-2f73-45d6-a267-f4256d66e13f\") " pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.239762 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.239746    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtkg7\" (UniqueName: \"kubernetes.io/projected/ac4fb66e-83c3-4cb2-81c0-07385c080c9b-kube-api-access-mtkg7\") pod \"iptables-alerter-bvxjr\" (UID: \"ac4fb66e-83c3-4cb2-81c0-07385c080c9b\") " pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.254767 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.254733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal" event={"ID":"a952e817f62df0bda7ddea87b49c2739","Type":"ContainerStarted","Data":"398996fa2ac18f863abe05c3b053ca7877e51714dd30b2764dfabbe0ad61c460"}
Jul 27 17:20:45.255835 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.255813    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal" event={"ID":"fdacb45223ffcd32a88e9773f54ccf6a","Type":"ContainerStarted","Data":"75b3c39774ed80dbeff1c2c1cdaec8ef0e21c3be95705ff6b175b39b72cd5591"}
Jul 27 17:20:45.326840 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.326820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nql6\" (UniqueName: \"kubernetes.io/projected/8688d57f-b060-4a55-a4aa-a503363dfe2c-kube-api-access-7nql6\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.326956 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.326846    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.326956 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.326863    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-socket-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.326956 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.326886    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-cnibin\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.326956 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.326918    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60f24a69-e287-4fc4-a659-e0d275e17215-konnectivity-ca\") pod \"konnectivity-agent-vhdzk\" (UID: \"60f24a69-e287-4fc4-a659-e0d275e17215\") " pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.327158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.326994    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-cnibin\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.327158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327041    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-socket-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327056    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.327158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327077    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.327158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327094    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60f24a69-e287-4fc4-a659-e0d275e17215-agent-certs\") pod \"konnectivity-agent-vhdzk\" (UID: \"60f24a69-e287-4fc4-a659-e0d275e17215\") " pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.327158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327156    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-device-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f91c41b-a2d4-4635-840f-09bf2302fd91-tmp-dir\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327210    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327229    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327236    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327237    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-device-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7r7r\" (UniqueName: \"kubernetes.io/projected/12fa5ebb-f199-4903-8d8d-d571608e044b-kube-api-access-x7r7r\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327360    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-sys-fs\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327385    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f91c41b-a2d4-4635-840f-09bf2302fd91-hosts-file\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tjp4\" (UniqueName: \"kubernetes.io/projected/4f91c41b-a2d4-4635-840f-09bf2302fd91-kube-api-access-8tjp4\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-os-release\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327439    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60f24a69-e287-4fc4-a659-e0d275e17215-konnectivity-ca\") pod \"konnectivity-agent-vhdzk\" (UID: \"60f24a69-e287-4fc4-a659-e0d275e17215\") " pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327443    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-sys-fs\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327519    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8pvv\" (UniqueName: \"kubernetes.io/projected/6b483d54-7e05-4952-82b2-e25106bd4549-kube-api-access-x8pvv\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327550    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-etc-selinux\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.327578 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327594    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-host\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327607    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-os-release\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327604    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327640    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-serviceca\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327678    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-host\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327688    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-registration-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327678    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4f91c41b-a2d4-4635-840f-09bf2302fd91-hosts-file\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327730    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-system-cni-dir\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327745    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b483d54-7e05-4952-82b2-e25106bd4549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8rhk\" (UniqueName: \"kubernetes.io/projected/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-kube-api-access-t8rhk\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327809    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b483d54-7e05-4952-82b2-e25106bd4549-system-cni-dir\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8688d57f-b060-4a55-a4aa-a503363dfe2c-registration-dir\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.327845    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.327896    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs podName:12fa5ebb-f199-4903-8d8d-d571608e044b nodeName:}" failed. No retries permitted until 2026-07-27 17:20:45.827878186 +0000 UTC m=+3.088853366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs") pod "network-metrics-daemon-dls6k" (UID: "12fa5ebb-f199-4903-8d8d-d571608e044b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:45.328199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.327997    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f91c41b-a2d4-4635-840f-09bf2302fd91-tmp-dir\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.328830 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.328814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-serviceca\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.329226 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.329211    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60f24a69-e287-4fc4-a659-e0d275e17215-agent-certs\") pod \"konnectivity-agent-vhdzk\" (UID: \"60f24a69-e287-4fc4-a659-e0d275e17215\") " pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.334675 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.334630    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tjp4\" (UniqueName: \"kubernetes.io/projected/4f91c41b-a2d4-4635-840f-09bf2302fd91-kube-api-access-8tjp4\") pod \"node-resolver-2kb9q\" (UID: \"4f91c41b-a2d4-4635-840f-09bf2302fd91\") " pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.335005 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.334987    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nql6\" (UniqueName: \"kubernetes.io/projected/8688d57f-b060-4a55-a4aa-a503363dfe2c-kube-api-access-7nql6\") pod \"aws-ebs-csi-driver-node-2k5qb\" (UID: \"8688d57f-b060-4a55-a4aa-a503363dfe2c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.336929 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.336910    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8rhk\" (UniqueName: \"kubernetes.io/projected/bfe4dd64-9586-428a-a704-c3d5cec3b3e7-kube-api-access-t8rhk\") pod \"node-ca-c7rx4\" (UID: \"bfe4dd64-9586-428a-a704-c3d5cec3b3e7\") " pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.337390 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.337375    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8pvv\" (UniqueName: \"kubernetes.io/projected/6b483d54-7e05-4952-82b2-e25106bd4549-kube-api-access-x8pvv\") pod \"multus-additional-cni-plugins-nfjn2\" (UID: \"6b483d54-7e05-4952-82b2-e25106bd4549\") " pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.337667 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.337646    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7r7r\" (UniqueName: \"kubernetes.io/projected/12fa5ebb-f199-4903-8d8d-d571608e044b-kube-api-access-x7r7r\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:45.408005 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.407983    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-czzk6"
Jul 27 17:20:45.414842 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.414822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-525fp"
Jul 27 17:20:45.416777 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:45.416751    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6e84a1_2f73_45d6_a267_f4256d66e13f.slice/crio-d61085277a6b0b0a6ec613caea00e6874a9e07af6c563f5dffde79e98995051c WatchSource:0}: Error finding container d61085277a6b0b0a6ec613caea00e6874a9e07af6c563f5dffde79e98995051c: Status 404 returned error can't find the container with id d61085277a6b0b0a6ec613caea00e6874a9e07af6c563f5dffde79e98995051c
Jul 27 17:20:45.424081 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:45.424037    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3408bd3e_f7d9_43a3_a172_f8c55e43e578.slice/crio-3894c8e1d9fb29eb7be7e76b9ac6d3cf5ef8cefe33cf0d97aed1da55e1af178c WatchSource:0}: Error finding container 3894c8e1d9fb29eb7be7e76b9ac6d3cf5ef8cefe33cf0d97aed1da55e1af178c: Status 404 returned error can't find the container with id 3894c8e1d9fb29eb7be7e76b9ac6d3cf5ef8cefe33cf0d97aed1da55e1af178c
Jul 27 17:20:45.424966 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.424946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvxjr"
Jul 27 17:20:45.430280 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.430259    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:20:45.434539 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.434517    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c7rx4"
Jul 27 17:20:45.437277 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:45.437227    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4fb66e_83c3_4cb2_81c0_07385c080c9b.slice/crio-10994c588b1f1e6c606801c4adc5b8394740df4f3428d14c480b64c02195189e WatchSource:0}: Error finding container 10994c588b1f1e6c606801c4adc5b8394740df4f3428d14c480b64c02195189e: Status 404 returned error can't find the container with id 10994c588b1f1e6c606801c4adc5b8394740df4f3428d14c480b64c02195189e
Jul 27 17:20:45.439964 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.439946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:20:45.445395 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:45.445348    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe4dd64_9586_428a_a704_c3d5cec3b3e7.slice/crio-c9ee0cf16db7b32a8b97262d812febdcc328ba1887ba6c5bc6f2087a74721807 WatchSource:0}: Error finding container c9ee0cf16db7b32a8b97262d812febdcc328ba1887ba6c5bc6f2087a74721807: Status 404 returned error can't find the container with id c9ee0cf16db7b32a8b97262d812febdcc328ba1887ba6c5bc6f2087a74721807
Jul 27 17:20:45.445488 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.445439    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb"
Jul 27 17:20:45.450749 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.450730    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2kb9q"
Jul 27 17:20:45.451932 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:45.451904    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f24a69_e287_4fc4_a659_e0d275e17215.slice/crio-11c852dbc4797eaaf13f44f6dc6b8b78d18211d035ba1cda2a2f500580a7b379 WatchSource:0}: Error finding container 11c852dbc4797eaaf13f44f6dc6b8b78d18211d035ba1cda2a2f500580a7b379: Status 404 returned error can't find the container with id 11c852dbc4797eaaf13f44f6dc6b8b78d18211d035ba1cda2a2f500580a7b379
Jul 27 17:20:45.454455 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.454317    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nfjn2"
Jul 27 17:20:45.465785 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:45.465764    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f91c41b_a2d4_4635_840f_09bf2302fd91.slice/crio-36a0056f182086e82dfe3e1f21cfb99f784d4eb60334c56c3883bb868dd3737f WatchSource:0}: Error finding container 36a0056f182086e82dfe3e1f21cfb99f784d4eb60334c56c3883bb868dd3737f: Status 404 returned error can't find the container with id 36a0056f182086e82dfe3e1f21cfb99f784d4eb60334c56c3883bb868dd3737f
Jul 27 17:20:45.469102 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:20:45.469078    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b483d54_7e05_4952_82b2_e25106bd4549.slice/crio-8728b3e406f1f94cf8160a92f838c4b1f5549724bfbde79c5091153f3c4ff362 WatchSource:0}: Error finding container 8728b3e406f1f94cf8160a92f838c4b1f5549724bfbde79c5091153f3c4ff362: Status 404 returned error can't find the container with id 8728b3e406f1f94cf8160a92f838c4b1f5549724bfbde79c5091153f3c4ff362
Jul 27 17:20:45.729897 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.729820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:45.730049 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.729970    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:45.730049 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.730043    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret podName:e4e307bf-7bb5-44dc-8dc0-890bbafffb78 nodeName:}" failed. No retries permitted until 2026-07-27 17:20:46.730013248 +0000 UTC m=+3.990988429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret") pod "global-pull-secret-syncer-l288b" (UID: "e4e307bf-7bb5-44dc-8dc0-890bbafffb78") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:45.831323 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.830848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:45.831323 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:45.830940    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:45.831323 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.831101    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:20:45.831323 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.831118    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:20:45.831323 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.831129    2565 projected.go:196] Error preparing data for projected volume kube-api-access-96k4k for pod openshift-network-diagnostics/network-check-target-bdtq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:45.831323 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.831185    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k podName:c0ec967f-de07-46cc-899c-c66c49a81afd nodeName:}" failed. No retries permitted until 2026-07-27 17:20:46.831167998 +0000 UTC m=+4.092143215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-96k4k" (UniqueName: "kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k") pod "network-check-target-bdtq8" (UID: "c0ec967f-de07-46cc-899c-c66c49a81afd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:45.831323 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.831310    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:45.831758 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:45.831370    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs podName:12fa5ebb-f199-4903-8d8d-d571608e044b nodeName:}" failed. No retries permitted until 2026-07-27 17:20:46.831352825 +0000 UTC m=+4.092328024 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs") pod "network-metrics-daemon-dls6k" (UID: "12fa5ebb-f199-4903-8d8d-d571608e044b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:46.160695 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.160542    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:15:45 +0000 UTC" deadline="2028-04-03 10:26:21.85503541 +0000 UTC"
Jul 27 17:20:46.160695 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.160596    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14777h5m35.694443932s"
Jul 27 17:20:46.249692 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.249655    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:46.249862 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.249791    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:46.266039 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.265966    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2kb9q" event={"ID":"4f91c41b-a2d4-4635-840f-09bf2302fd91","Type":"ContainerStarted","Data":"36a0056f182086e82dfe3e1f21cfb99f784d4eb60334c56c3883bb868dd3737f"}
Jul 27 17:20:46.271425 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.271392    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb" event={"ID":"8688d57f-b060-4a55-a4aa-a503363dfe2c","Type":"ContainerStarted","Data":"d68a63f4b97dd1b4244a519120ada6c105953ea0d411b03b107c2613031bdf3a"}
Jul 27 17:20:46.277999 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.277899    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhdzk" event={"ID":"60f24a69-e287-4fc4-a659-e0d275e17215","Type":"ContainerStarted","Data":"11c852dbc4797eaaf13f44f6dc6b8b78d18211d035ba1cda2a2f500580a7b379"}
Jul 27 17:20:46.284703 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.284675    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c7rx4" event={"ID":"bfe4dd64-9586-428a-a704-c3d5cec3b3e7","Type":"ContainerStarted","Data":"c9ee0cf16db7b32a8b97262d812febdcc328ba1887ba6c5bc6f2087a74721807"}
Jul 27 17:20:46.288039 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.288016    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"bc4c9bffb3bd80374e79c05b6a43d46b641161ffa316a40e2e8d01373c319f6e"}
Jul 27 17:20:46.292329 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.292241    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvxjr" event={"ID":"ac4fb66e-83c3-4cb2-81c0-07385c080c9b","Type":"ContainerStarted","Data":"10994c588b1f1e6c606801c4adc5b8394740df4f3428d14c480b64c02195189e"}
Jul 27 17:20:46.300452 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.300426    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-czzk6" event={"ID":"4e6e84a1-2f73-45d6-a267-f4256d66e13f","Type":"ContainerStarted","Data":"d61085277a6b0b0a6ec613caea00e6874a9e07af6c563f5dffde79e98995051c"}
Jul 27 17:20:46.303841 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.303815    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerStarted","Data":"8728b3e406f1f94cf8160a92f838c4b1f5549724bfbde79c5091153f3c4ff362"}
Jul 27 17:20:46.306116 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.306092    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-525fp" event={"ID":"3408bd3e-f7d9-43a3-a172-f8c55e43e578","Type":"ContainerStarted","Data":"3894c8e1d9fb29eb7be7e76b9ac6d3cf5ef8cefe33cf0d97aed1da55e1af178c"}
Jul 27 17:20:46.739859 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.739242    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:46.739859 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.739419    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:46.739859 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.739481    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret podName:e4e307bf-7bb5-44dc-8dc0-890bbafffb78 nodeName:}" failed. No retries permitted until 2026-07-27 17:20:48.739462483 +0000 UTC m=+6.000437662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret") pod "global-pull-secret-syncer-l288b" (UID: "e4e307bf-7bb5-44dc-8dc0-890bbafffb78") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:46.839667 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.839630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:46.839868 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:46.839703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:46.839868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.839854    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:20:46.839984 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.839872    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:20:46.839984 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.839885    2565 projected.go:196] Error preparing data for projected volume kube-api-access-96k4k for pod openshift-network-diagnostics/network-check-target-bdtq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:46.839984 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.839940    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k podName:c0ec967f-de07-46cc-899c-c66c49a81afd nodeName:}" failed. No retries permitted until 2026-07-27 17:20:48.839922909 +0000 UTC m=+6.100898101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-96k4k" (UniqueName: "kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k") pod "network-check-target-bdtq8" (UID: "c0ec967f-de07-46cc-899c-c66c49a81afd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:46.840362 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.840343    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:46.840453 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:46.840399    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs podName:12fa5ebb-f199-4903-8d8d-d571608e044b nodeName:}" failed. No retries permitted until 2026-07-27 17:20:48.840383597 +0000 UTC m=+6.101358789 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs") pod "network-metrics-daemon-dls6k" (UID: "12fa5ebb-f199-4903-8d8d-d571608e044b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:47.161397 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:47.161307    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:15:45 +0000 UTC" deadline="2028-01-18 14:11:12.604144122 +0000 UTC"
Jul 27 17:20:47.161397 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:47.161347    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12956h50m25.442800896s"
Jul 27 17:20:47.252024 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:47.251430    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:47.252024 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:47.251597    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:47.252275 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:47.252051    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:47.252275 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:47.252168    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:20:48.249855 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:48.249820    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:48.250355 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.249953    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:48.756110 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:48.756060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:48.756288 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.756229    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:48.756288 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.756307    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret podName:e4e307bf-7bb5-44dc-8dc0-890bbafffb78 nodeName:}" failed. No retries permitted until 2026-07-27 17:20:52.756273527 +0000 UTC m=+10.017248708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret") pod "global-pull-secret-syncer-l288b" (UID: "e4e307bf-7bb5-44dc-8dc0-890bbafffb78") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:48.856858 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:48.856818    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:48.857075 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:48.856889    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:48.857075 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.857029    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:20:48.857075 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.857046    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:20:48.857075 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.857059    2565 projected.go:196] Error preparing data for projected volume kube-api-access-96k4k for pod openshift-network-diagnostics/network-check-target-bdtq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:48.857338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.857111    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k podName:c0ec967f-de07-46cc-899c-c66c49a81afd nodeName:}" failed. No retries permitted until 2026-07-27 17:20:52.85709422 +0000 UTC m=+10.118069397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-96k4k" (UniqueName: "kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k") pod "network-check-target-bdtq8" (UID: "c0ec967f-de07-46cc-899c-c66c49a81afd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:48.857338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.857308    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:48.857455 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:48.857369    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs podName:12fa5ebb-f199-4903-8d8d-d571608e044b nodeName:}" failed. No retries permitted until 2026-07-27 17:20:52.857352541 +0000 UTC m=+10.118327753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs") pod "network-metrics-daemon-dls6k" (UID: "12fa5ebb-f199-4903-8d8d-d571608e044b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:49.249989 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:49.249908    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:49.250446 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:49.250038    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:49.250522 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:49.250475    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:49.250610 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:49.250585    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:20:50.250196 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:50.249630    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:50.250196 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:50.249829    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:51.250240 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:51.250204    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:51.250650 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:51.250211    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:51.250650 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:51.250354    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:51.250650 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:51.250432    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:20:52.249985 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:52.249901    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:52.250151 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.250048    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:52.789640 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:52.789594    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:52.790090 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.789731    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:52.790090 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.789802    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret podName:e4e307bf-7bb5-44dc-8dc0-890bbafffb78 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:00.789783029 +0000 UTC m=+18.050758209 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret") pod "global-pull-secret-syncer-l288b" (UID: "e4e307bf-7bb5-44dc-8dc0-890bbafffb78") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:20:52.890663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:52.890606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:52.890830 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:52.890676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:52.890894 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.890838    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:20:52.890894 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.890856    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:20:52.890894 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.890876    2565 projected.go:196] Error preparing data for projected volume kube-api-access-96k4k for pod openshift-network-diagnostics/network-check-target-bdtq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:52.891038 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.890928    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k podName:c0ec967f-de07-46cc-899c-c66c49a81afd nodeName:}" failed. No retries permitted until 2026-07-27 17:21:00.890910415 +0000 UTC m=+18.151885592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-96k4k" (UniqueName: "kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k") pod "network-check-target-bdtq8" (UID: "c0ec967f-de07-46cc-899c-c66c49a81afd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:20:52.891359 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.891328    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:52.891454 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:52.891379    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs podName:12fa5ebb-f199-4903-8d8d-d571608e044b nodeName:}" failed. No retries permitted until 2026-07-27 17:21:00.891362832 +0000 UTC m=+18.152338012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs") pod "network-metrics-daemon-dls6k" (UID: "12fa5ebb-f199-4903-8d8d-d571608e044b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:20:53.250550 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:53.250475    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:53.250550 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:53.250493    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:53.252071 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:53.251607    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:53.252071 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:53.252033    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:20:54.250161 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:54.250126    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:54.250631 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:54.250259    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:55.250541 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:55.250509    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:55.250911 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:55.250647    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:20:55.250911 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:55.250685    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:55.250911 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:55.250756    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:56.249826 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:56.249764    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:56.249947 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:56.249853    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:57.251005 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:57.250663    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:57.251005 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:57.250724    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:57.251005 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:57.250832    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:20:57.251005 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:57.250918    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:58.250501 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:58.250462    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:20:58.250682 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:58.250598    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:20:59.250353 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:59.250317    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:20:59.250797 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:20:59.250329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:20:59.250797 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:59.250448    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:20:59.250797 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:20:59.250534    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:00.249944 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:00.249914    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:00.250075 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.250008    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:00.853328 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:00.853275    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:00.853868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.853429    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:00.853868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.853503    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret podName:e4e307bf-7bb5-44dc-8dc0-890bbafffb78 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:16.853484673 +0000 UTC m=+34.114459849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret") pod "global-pull-secret-syncer-l288b" (UID: "e4e307bf-7bb5-44dc-8dc0-890bbafffb78") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:00.953939 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:00.953908    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:00.954132 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:00.953968    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:00.954132 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.954059    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:00.954132 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.954077    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:00.954132 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.954096    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:00.954132 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.954108    2565 projected.go:196] Error preparing data for projected volume kube-api-access-96k4k for pod openshift-network-diagnostics/network-check-target-bdtq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:00.954132 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.954116    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs podName:12fa5ebb-f199-4903-8d8d-d571608e044b nodeName:}" failed. No retries permitted until 2026-07-27 17:21:16.954098487 +0000 UTC m=+34.215073677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs") pod "network-metrics-daemon-dls6k" (UID: "12fa5ebb-f199-4903-8d8d-d571608e044b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:00.954463 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:00.954143    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k podName:c0ec967f-de07-46cc-899c-c66c49a81afd nodeName:}" failed. No retries permitted until 2026-07-27 17:21:16.954133316 +0000 UTC m=+34.215108508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-96k4k" (UniqueName: "kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k") pod "network-check-target-bdtq8" (UID: "c0ec967f-de07-46cc-899c-c66c49a81afd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:01.250311 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:01.250216    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:01.250311 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:01.250216    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:01.250526 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:01.250372    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:01.250589 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:01.250517    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:02.250021 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:02.249983    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:02.250461 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:02.250098    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:03.251089 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:03.251056    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:03.251556 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:03.251146    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:03.251556 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:03.251272    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:03.251556 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:03.251415    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:04.249884 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:04.249853    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:04.250031 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:04.249963    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:05.250237 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.249914    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:05.250758 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:05.250388    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:05.250758 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.249982    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:05.250758 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:05.250516    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:05.346600 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.346567    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal" event={"ID":"a952e817f62df0bda7ddea87b49c2739","Type":"ContainerStarted","Data":"49e50f9bc8a76f320a3faa0daaa00056bfb3be883597973bc321de055827997a"}
Jul 27 17:21:05.347767 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.347745    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2kb9q" event={"ID":"4f91c41b-a2d4-4635-840f-09bf2302fd91","Type":"ContainerStarted","Data":"5ae0ea326dd76168be6de5223d0121d78cc7e03b7d4040345535d8945521e82e"}
Jul 27 17:21:05.349057 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.349026    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb" event={"ID":"8688d57f-b060-4a55-a4aa-a503363dfe2c","Type":"ContainerStarted","Data":"d7ed14b4fc0fa03ee1794480e1f3deef9276d212c3b8a83f680e058fa8e709ed"}
Jul 27 17:21:05.350099 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.350077    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhdzk" event={"ID":"60f24a69-e287-4fc4-a659-e0d275e17215","Type":"ContainerStarted","Data":"a544e8d925d7a39fdef1327ee74a1e1f650bee4331c59e7d97f744e5927ca26c"}
Jul 27 17:21:05.351356 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.351337    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c7rx4" event={"ID":"bfe4dd64-9586-428a-a704-c3d5cec3b3e7","Type":"ContainerStarted","Data":"342a5352d49e6f437acc4c1bb8544f9bf110885c4163f93ff422980a5ef1fa11"}
Jul 27 17:21:05.352824 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.352804    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"e11a7c58d27ee852a6f6ed4c49984e7e0833e4ceb530737db14359b0e2ed28b0"}
Jul 27 17:21:05.352949 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.352865    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"1168b6dda43887f72fdf1939fb63dcc37b02b7b7acc348ecddf8c550b5f3cb41"}
Jul 27 17:21:05.354065 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.354045    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-czzk6" event={"ID":"4e6e84a1-2f73-45d6-a267-f4256d66e13f","Type":"ContainerStarted","Data":"1fa079f6879740e0a59741d97d95b8947b6d101be80dfcf29e472250ff5c6d16"}
Jul 27 17:21:05.355409 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.355389    2565 generic.go:356] "Generic (PLEG): container finished" podID="fdacb45223ffcd32a88e9773f54ccf6a" containerID="dc975c3e732864304646106b08399f9a324f64f5a65e4f1059b7fb71b120eaa6" exitCode=0
Jul 27 17:21:05.355492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.355454    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal" event={"ID":"fdacb45223ffcd32a88e9773f54ccf6a","Type":"ContainerDied","Data":"dc975c3e732864304646106b08399f9a324f64f5a65e4f1059b7fb71b120eaa6"}
Jul 27 17:21:05.356683 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.356665    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerStarted","Data":"9fb1d3a0f324852abafe14d40bd27718b21812563834c444c45e55f2dbe541a0"}
Jul 27 17:21:05.357972 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.357954    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-525fp" event={"ID":"3408bd3e-f7d9-43a3-a172-f8c55e43e578","Type":"ContainerStarted","Data":"6da78e8d78f13ca00fb9c7d250d7b56fb55231bee8ae6ebb68fda87a1eeead69"}
Jul 27 17:21:05.362431 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.362394    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-8.ec2.internal" podStartSLOduration=21.362384209 podStartE2EDuration="21.362384209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:05.362377759 +0000 UTC m=+22.623352958" watchObservedRunningTime="2026-07-27 17:21:05.362384209 +0000 UTC m=+22.623359407"
Jul 27 17:21:05.376436 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.376401    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vhdzk" podStartSLOduration=3.3663596350000002 podStartE2EDuration="22.376388071s" totalImagesPullingTime="19.010028436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:05.376284084 +0000 UTC m=+22.637259282" watchObservedRunningTime="2026-07-27 17:21:05.376388071 +0000 UTC m=+22.637363273"
Jul 27 17:21:05.412343 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.412283    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2kb9q" podStartSLOduration=3.384439586 podStartE2EDuration="22.412269054s" totalImagesPullingTime="19.027829468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:05.390683759 +0000 UTC m=+22.651658956" watchObservedRunningTime="2026-07-27 17:21:05.412269054 +0000 UTC m=+22.673244252"
Jul 27 17:21:05.428683 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.428651    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-525fp" podStartSLOduration=3.336242383 podStartE2EDuration="22.428639167s" totalImagesPullingTime="19.092396784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:05.42838525 +0000 UTC m=+22.689360448" watchObservedRunningTime="2026-07-27 17:21:05.428639167 +0000 UTC m=+22.689614363"
Jul 27 17:21:05.469199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.469164    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-czzk6" podStartSLOduration=3.424753285 podStartE2EDuration="22.46914795s" totalImagesPullingTime="19.044394665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:05.468651121 +0000 UTC m=+22.729626320" watchObservedRunningTime="2026-07-27 17:21:05.46914795 +0000 UTC m=+22.730123169"
Jul 27 17:21:05.469285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:05.469272    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c7rx4" podStartSLOduration=3.8174167580000002 podStartE2EDuration="22.469266374s" totalImagesPullingTime="18.651849616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:05.445228109 +0000 UTC m=+22.706203308" watchObservedRunningTime="2026-07-27 17:21:05.469266374 +0000 UTC m=+22.730241572"
Jul 27 17:21:06.157241 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.157063    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:21:06.250637 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.250610    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:06.251075 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:06.250699    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:06.361521 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.361490    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b483d54-7e05-4952-82b2-e25106bd4549" containerID="9fb1d3a0f324852abafe14d40bd27718b21812563834c444c45e55f2dbe541a0" exitCode=0
Jul 27 17:21:06.361702 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.361575    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerDied","Data":"9fb1d3a0f324852abafe14d40bd27718b21812563834c444c45e55f2dbe541a0"}
Jul 27 17:21:06.364628 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.364604    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"3fcf56620c3ea71b2261a1a20f4efe092f8309ff2a050a9b17b0d0b4ed673d7f"}
Jul 27 17:21:06.364739 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.364678    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"c4ed98e9ad59be89d866dee4f80d3a2e78f2a64070eac351db2175c5deac38f0"}
Jul 27 17:21:06.364739 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.364730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"a9bc3cb4fe7dbb4e1120ab1bca44ed73fbe9d98b32b2db08637dfc3a887c809d"}
Jul 27 17:21:06.364842 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.364776    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"97d6ed4d041ff43b3f3ae229f147fe3409274196c17e5fb0441021c259606a8b"}
Jul 27 17:21:06.365843 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.365814    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvxjr" event={"ID":"ac4fb66e-83c3-4cb2-81c0-07385c080c9b","Type":"ContainerStarted","Data":"cc29f3fab276b7448fb6b8e766c98d0ccd6e354b05117fd23e198ce7c65e1182"}
Jul 27 17:21:06.417870 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.417786    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bvxjr" podStartSLOduration=4.759674757 podStartE2EDuration="23.417772059s" totalImagesPullingTime="18.658097302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:06.41741642 +0000 UTC m=+23.678391618" watchObservedRunningTime="2026-07-27 17:21:06.417772059 +0000 UTC m=+23.678747367"
Jul 27 17:21:06.541161 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:06.541140    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 17:21:07.180975 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.180839    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T17:21:06.541156614Z","UUID":"dce9a004-b3cc-48ec-b721-a5936fad72d8","Handler":null,"Name":"","Endpoint":""}
Jul 27 17:21:07.182959 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.182935    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 17:21:07.183112 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.182981    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 17:21:07.249899 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.249874    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:07.250013 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:07.249972    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:07.250013 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.249998    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:07.250095 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:07.250067    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:07.370428 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.370375    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal" event={"ID":"fdacb45223ffcd32a88e9773f54ccf6a","Type":"ContainerStarted","Data":"e4bdd243adbc9ef9d4d76ab6405bdf547c45dbdcf2c5ec214c8ef17ac852a809"}
Jul 27 17:21:07.372271 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.372248    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb" event={"ID":"8688d57f-b060-4a55-a4aa-a503363dfe2c","Type":"ContainerStarted","Data":"44b0202552216e083e4cc2b748c15211a33b5aa75e3e26921ce4a99c8d9b4b42"}
Jul 27 17:21:07.395797 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.395670    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-8.ec2.internal" podStartSLOduration=23.395652466 podStartE2EDuration="23.395652466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:07.395401457 +0000 UTC m=+24.656376657" watchObservedRunningTime="2026-07-27 17:21:07.395652466 +0000 UTC m=+24.656627666"
Jul 27 17:21:07.451960 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.451865    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:21:07.452382 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:07.452365    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:21:08.250682 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:08.250598    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:08.250835 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:08.250735    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:08.377406 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:08.377368    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb" event={"ID":"8688d57f-b060-4a55-a4aa-a503363dfe2c","Type":"ContainerStarted","Data":"72c0199feea371a540a7663b53f34a74f0e82c73a09db571f5586d375001bc65"}
Jul 27 17:21:08.381648 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:08.381625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"31f2d655cef747e0102048fb341dae6e0ec15f7a74cbbd4957ccb4b3399d7d5e"}
Jul 27 17:21:08.382374 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:08.382348    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vhdzk"
Jul 27 17:21:08.395663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:08.395613    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2k5qb" podStartSLOduration=4.231214251 podStartE2EDuration="25.395599548s" totalImagesPullingTime="21.164385297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:08.395372684 +0000 UTC m=+25.656347884" watchObservedRunningTime="2026-07-27 17:21:08.395599548 +0000 UTC m=+25.656574746"
Jul 27 17:21:09.249718 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:09.249694    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:09.249878 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:09.249694    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:09.249878 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:09.249804    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:09.249878 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:09.249869    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:10.249989 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.249707    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:10.250525 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:10.250033    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:10.395531 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.395491    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" event={"ID":"b125957e-6328-4af6-bc35-ed60ae66dc7e","Type":"ContainerStarted","Data":"f523f09fe19f2be536c0c46a8f3e930d9fb0bd4ada17cd9c5432c625c7489968"}
Jul 27 17:21:10.395866 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.395845    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:21:10.395970 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.395931    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:21:10.396039 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.395978    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:21:10.412794 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.412766    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:21:10.413228 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.413208    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:21:10.427531 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:10.427473    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4" podStartSLOduration=7.789402719 podStartE2EDuration="27.427456291s" totalImagesPullingTime="19.638053572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:10.427385251 +0000 UTC m=+27.688360451" watchObservedRunningTime="2026-07-27 17:21:10.427456291 +0000 UTC m=+27.688431491"
Jul 27 17:21:11.250736 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.250590    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:11.251429 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:11.250741    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:11.251429 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.250819    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:11.251429 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:11.250929    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:11.970384 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.970196    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l288b"]
Jul 27 17:21:11.970492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.970480    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:11.970597 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:11.970578    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:11.973687 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.973660    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bdtq8"]
Jul 27 17:21:11.973785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.973737    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:11.973855 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:11.973806    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:11.986838 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.986811    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dls6k"]
Jul 27 17:21:11.986938 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:11.986912    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:11.987023 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:11.987004    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:12.401479 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:12.401449    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b483d54-7e05-4952-82b2-e25106bd4549" containerID="1301f5aae366868472831d2597ce8a9397965936bf9e96383f33840f570a64d9" exitCode=0
Jul 27 17:21:12.401974 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:12.401525    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerDied","Data":"1301f5aae366868472831d2597ce8a9397965936bf9e96383f33840f570a64d9"}
Jul 27 17:21:13.250584 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:13.250557    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:13.250717 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:13.250634    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:14.250690 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:14.250612    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:14.251111 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:14.250612    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:14.251111 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:14.250744    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:14.251111 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:14.250799    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:14.410353 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:14.410318    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b483d54-7e05-4952-82b2-e25106bd4549" containerID="24d52ea2682afd618e0acabda81d93ff72230cb268ea54cbb30a3b3cf65bf5c1" exitCode=0
Jul 27 17:21:14.410498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:14.410359    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerDied","Data":"24d52ea2682afd618e0acabda81d93ff72230cb268ea54cbb30a3b3cf65bf5c1"}
Jul 27 17:21:15.250668 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:15.250639    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:15.250867 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:15.250742    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:16.249846 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:16.249790    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:16.249846 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:16.249814    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:16.250074 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.249910    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l288b" podUID="e4e307bf-7bb5-44dc-8dc0-890bbafffb78"
Jul 27 17:21:16.250074 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.250054    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dls6k" podUID="12fa5ebb-f199-4903-8d8d-d571608e044b"
Jul 27 17:21:16.877261 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:16.877239    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:16.877545 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.877360    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:16.877545 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.877406    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret podName:e4e307bf-7bb5-44dc-8dc0-890bbafffb78 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:48.877393022 +0000 UTC m=+66.138368199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret") pod "global-pull-secret-syncer-l288b" (UID: "e4e307bf-7bb5-44dc-8dc0-890bbafffb78") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:21:16.978456 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:16.978433    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:16.978569 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:16.978478    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:16.978626 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.978587    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:21:16.978626 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.978588    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:16.978626 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.978600    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:21:16.978626 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.978611    2565 projected.go:196] Error preparing data for projected volume kube-api-access-96k4k for pod openshift-network-diagnostics/network-check-target-bdtq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:16.978778 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.978659    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs podName:12fa5ebb-f199-4903-8d8d-d571608e044b nodeName:}" failed. No retries permitted until 2026-07-27 17:21:48.978639454 +0000 UTC m=+66.239614651 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs") pod "network-metrics-daemon-dls6k" (UID: "12fa5ebb-f199-4903-8d8d-d571608e044b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:21:16.978778 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:16.978682    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k podName:c0ec967f-de07-46cc-899c-c66c49a81afd nodeName:}" failed. No retries permitted until 2026-07-27 17:21:48.978670819 +0000 UTC m=+66.239646008 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-96k4k" (UniqueName: "kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k") pod "network-check-target-bdtq8" (UID: "c0ec967f-de07-46cc-899c-c66c49a81afd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:21:17.250208 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.250136    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:17.250370 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.250263    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdtq8" podUID="c0ec967f-de07-46cc-899c-c66c49a81afd"
Jul 27 17:21:17.419588 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.419558    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b483d54-7e05-4952-82b2-e25106bd4549" containerID="0c0f4fd17f1e917d2ade8ab13b731aea2c153174e7c999b4ef8bd2543b4649ff" exitCode=0
Jul 27 17:21:17.419697 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.419609    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerDied","Data":"0c0f4fd17f1e917d2ade8ab13b731aea2c153174e7c999b4ef8bd2543b4649ff"}
Jul 27 17:21:17.564732 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.564710    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeReady"
Jul 27 17:21:17.564846 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.564806    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 17:21:17.609651 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.609621    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86fbf99f7-pzptv"]
Jul 27 17:21:17.625512 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.625489    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mqz4z"]
Jul 27 17:21:17.625633 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.625608    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.640331 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.640307    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"]
Jul 27 17:21:17.640434 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.640366    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.641416 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.641398    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 17:21:17.643678 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.643658    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 17:21:17.643870 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.643758    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 17:21:17.643870 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.643848    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rtxtk\""
Jul 27 17:21:17.646245 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.646226    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.646514 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.646418    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wnjmr\""
Jul 27 17:21:17.647022 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.647001    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.658536 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.658249    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 17:21:17.659270 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.659251    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"]
Jul 27 17:21:17.659469 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.659451    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:17.669818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.669796    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 17:21:17.669953 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.669915    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.669953 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.669915    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.671805 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.671786    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xjqb6"]
Jul 27 17:21:17.671926 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.671911    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.672374 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.672357    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 17:21:17.674106 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.674082    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bvxqm\""
Jul 27 17:21:17.675015 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.674999    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 17:21:17.675729 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.675711    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 17:21:17.675991 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.675973    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.676085 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.676071    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-k55f8\""
Jul 27 17:21:17.678257 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.678239    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 17:21:17.680823 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.680804    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 17:21:17.684846 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.684826    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.693150 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.693131    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"]
Jul 27 17:21:17.693249 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.693198    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.696110 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.696091    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 17:21:17.699335 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.699314    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.699617 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.699602    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 17:21:17.704144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.703872    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rf7vh\""
Jul 27 17:21:17.707211 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.707196    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.711491 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.711474    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58fd4bb687-9jltm"]
Jul 27 17:21:17.711607 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.711592    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"
Jul 27 17:21:17.715015 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.714997    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.715270 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.715252    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 17:21:17.717532 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.717512    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8fm7v\""
Jul 27 17:21:17.718117 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.718104    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.723781 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.723763    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"]
Jul 27 17:21:17.723874 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.723860    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.726371 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.726353    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 17:21:17.726371 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.726365    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 17:21:17.726517 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.726373    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4fhbz\""
Jul 27 17:21:17.726517 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.726380    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 17:21:17.726707 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.726693    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.726767 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.726751    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.726822 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.726788    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 17:21:17.738276 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.738244    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"]
Jul 27 17:21:17.738369 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.738332    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.740850 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.740835    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 17:21:17.741478 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.741453    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 17:21:17.741561 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.741496    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6dfng\""
Jul 27 17:21:17.741561 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.741545    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.741671 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.741661    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.754630 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.754612    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86fbf99f7-pzptv"]
Jul 27 17:21:17.754704 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.754647    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"]
Jul 27 17:21:17.754704 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.754672    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.757502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.757484    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.757686 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.757669    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.757772 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.757716    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 17:21:17.757861 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.757816    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 17:21:17.758373 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.758354    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kjpq5\""
Jul 27 17:21:17.768884 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.768867    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"]
Jul 27 17:21:17.768988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.768975    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:17.772017 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.772001    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gv7q2\""
Jul 27 17:21:17.772087 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.772063    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 27 17:21:17.772374 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.772361    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 27 17:21:17.784095 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784078    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/76f6fe48-c826-4af3-bc3d-ab70010dcb93-snapshots\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.784172 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784102    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fphqz\" (UniqueName: \"kubernetes.io/projected/5410afce-d2a3-4357-9959-fd06d74ec212-kube-api-access-fphqz\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:17.784172 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb075801-f23d-4b07-a3e8-35b0dc2bda86-serving-cert\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.784172 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784152    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-image-registry-private-configuration\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784210    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33096a88-bbfe-4519-b239-157fa4382b21-ca-trust-extracted\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784228    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:17.784285 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/92430eb1-2164-4d31-90dc-d80938eaacb7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.784451 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784284    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76f6fe48-c826-4af3-bc3d-ab70010dcb93-tmp\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.784451 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784381    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb075801-f23d-4b07-a3e8-35b0dc2bda86-trusted-ca\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.784451 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784421    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.784577 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6fe48-c826-4af3-bc3d-ab70010dcb93-service-ca-bundle\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.784577 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784494    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6fe48-c826-4af3-bc3d-ab70010dcb93-serving-cert\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.784577 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784521    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5tf4\" (UniqueName: \"kubernetes.io/projected/fb075801-f23d-4b07-a3e8-35b0dc2bda86-kube-api-access-r5tf4\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.784577 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784547    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-trusted-ca\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784715 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784580    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-registry-certificates\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784715 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784603    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-installation-pull-secrets\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784715 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784620    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdsb7\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-kube-api-access-kdsb7\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784715 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784637    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb075801-f23d-4b07-a3e8-35b0dc2bda86-config\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.784715 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784651    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-bound-sa-token\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784715 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784688    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.784906 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784730    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6fe48-c826-4af3-bc3d-ab70010dcb93-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.784906 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4lgq\" (UniqueName: \"kubernetes.io/projected/76f6fe48-c826-4af3-bc3d-ab70010dcb93-kube-api-access-s4lgq\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.784906 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.784764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvmh7\" (UniqueName: \"kubernetes.io/projected/92430eb1-2164-4d31-90dc-d80938eaacb7-kube-api-access-gvmh7\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.785131 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.785118    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mqz4z"]
Jul 27 17:21:17.785178 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.785141    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xxzzn"]
Jul 27 17:21:17.785249 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.785235    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"
Jul 27 17:21:17.788045 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.788031    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.788132 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.788073    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.788268 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.788252    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pc4tx\""
Jul 27 17:21:17.797324 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.797305    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7r696"]
Jul 27 17:21:17.797403 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.797362    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.799982 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.799965    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 17:21:17.800073 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.799989    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 17:21:17.800073 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.800032    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2k848\""
Jul 27 17:21:17.805160 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805143    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"]
Jul 27 17:21:17.805239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805172    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xjqb6"]
Jul 27 17:21:17.805239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805185    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"]
Jul 27 17:21:17.805239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805198    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"]
Jul 27 17:21:17.805239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805215    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"]
Jul 27 17:21:17.805239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805227    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"]
Jul 27 17:21:17.805239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805239    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xxzzn"]
Jul 27 17:21:17.805518 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805254    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"]
Jul 27 17:21:17.805518 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805266    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58fd4bb687-9jltm"]
Jul 27 17:21:17.805518 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7r696"]
Jul 27 17:21:17.805518 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805287    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"]
Jul 27 17:21:17.805518 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.805360    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:17.808372 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.808355    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 17:21:17.808782 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.808769    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 17:21:17.809106 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.809067    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 17:21:17.809212 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.809117    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bgcdf\""
Jul 27 17:21:17.839206 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.839141    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhdzk_60f24a69-e287-4fc4-a659-e0d275e17215/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:17.885498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885512    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76f6fe48-c826-4af3-bc3d-ab70010dcb93-tmp\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885542    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lc7v\" (UniqueName: \"kubernetes.io/projected/9d428abe-51d0-42d3-b610-0552dd60f476-kube-api-access-4lc7v\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885641    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2kxq\" (UniqueName: \"kubernetes.io/projected/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-kube-api-access-k2kxq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b428v\" (UniqueName: \"kubernetes.io/projected/123a82b2-e990-49bd-897a-4628586d8e49-kube-api-access-b428v\") pod \"volume-data-source-validator-549c6d54cf-fmm85\" (UID: \"123a82b2-e990-49bd-897a-4628586d8e49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885692    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6fe48-c826-4af3-bc3d-ab70010dcb93-service-ca-bundle\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885735    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6fe48-c826-4af3-bc3d-ab70010dcb93-serving-cert\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5tf4\" (UniqueName: \"kubernetes.io/projected/fb075801-f23d-4b07-a3e8-35b0dc2bda86-kube-api-access-r5tf4\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-registry-certificates\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.885883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885812    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-installation-pull-secrets\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.885895    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76f6fe48-c826-4af3-bc3d-ab70010dcb93-tmp\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886214    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb075801-f23d-4b07-a3e8-35b0dc2bda86-config\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886228    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6fe48-c826-4af3-bc3d-ab70010dcb93-service-ca-bundle\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886248    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886280    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ccqr\" (UniqueName: \"kubernetes.io/projected/ee9b370c-7951-4a22-89bc-596ef49a2ad4-kube-api-access-4ccqr\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886328    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4lgq\" (UniqueName: \"kubernetes.io/projected/76f6fe48-c826-4af3-bc3d-ab70010dcb93-kube-api-access-s4lgq\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886352    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fphqz\" (UniqueName: \"kubernetes.io/projected/5410afce-d2a3-4357-9959-fd06d74ec212-kube-api-access-fphqz\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:17.886375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886372    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-default-certificate\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886398    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6fe48-c826-4af3-bc3d-ab70010dcb93-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886427    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tdd7\" (UniqueName: \"kubernetes.io/projected/3ffca930-1a03-46d2-9277-48017eeb1a11-kube-api-access-9tdd7\") pod \"network-check-source-7d88cb55d8-wqszm\" (UID: \"3ffca930-1a03-46d2-9277-48017eeb1a11\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886451    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9b370c-7951-4a22-89bc-596ef49a2ad4-config-volume\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886474    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb075801-f23d-4b07-a3e8-35b0dc2bda86-serving-cert\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886496    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-image-registry-private-configuration\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886522    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9b370c-7951-4a22-89bc-596ef49a2ad4-tmp-dir\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886600    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-registry-certificates\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886637    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/92430eb1-2164-4d31-90dc-d80938eaacb7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886697    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886724    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.886754 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb075801-f23d-4b07-a3e8-35b0dc2bda86-trusted-ca\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-trusted-ca\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886802    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdsb7\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-kube-api-access-kdsb7\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-stats-auth\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886842    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2529\" (UniqueName: \"kubernetes.io/projected/668f942d-f6dc-4152-9bd0-870aa27229ba-kube-api-access-q2529\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886876    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-bound-sa-token\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886916    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886948    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmh7\" (UniqueName: \"kubernetes.io/projected/92430eb1-2164-4d31-90dc-d80938eaacb7-kube-api-access-gvmh7\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/668f942d-f6dc-4152-9bd0-870aa27229ba-config\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.886988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghqdj\" (UniqueName: \"kubernetes.io/projected/29645a34-e258-423f-acbd-aa709a57ffd7-kube-api-access-ghqdj\") pod \"ingress-canary-7r696\" (UID: \"29645a34-e258-423f-acbd-aa709a57ffd7\") " pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887014    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/76f6fe48-c826-4af3-bc3d-ab70010dcb93-snapshots\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887054    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33096a88-bbfe-4519-b239-157fa4382b21-ca-trust-extracted\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887078    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887099    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.887318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887121    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/668f942d-f6dc-4152-9bd0-870aa27229ba-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.887936 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887145    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29645a34-e258-423f-acbd-aa709a57ffd7-cert\") pod \"ingress-canary-7r696\" (UID: \"29645a34-e258-423f-acbd-aa709a57ffd7\") " pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:17.887936 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887169    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.887936 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887188    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76cab81e-4e12-45f8-b53c-2134434c8663-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:17.887936 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887433    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb075801-f23d-4b07-a3e8-35b0dc2bda86-config\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.887936 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.887444    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/92430eb1-2164-4d31-90dc-d80938eaacb7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.888101    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb075801-f23d-4b07-a3e8-35b0dc2bda86-trusted-ca\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.888168    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.888279    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.888316    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86fbf99f7-pzptv: secret "image-registry-tls" not found
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.888338    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls podName:5410afce-d2a3-4357-9959-fd06d74ec212 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:18.388318868 +0000 UTC m=+35.649294062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls") pod "cluster-samples-operator-6748489656-l8qgt" (UID: "5410afce-d2a3-4357-9959-fd06d74ec212") : secret "samples-operator-tls" not found
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.888363    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls podName:33096a88-bbfe-4519-b239-157fa4382b21 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:18.38834779 +0000 UTC m=+35.649322974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls") pod "image-registry-86fbf99f7-pzptv" (UID: "33096a88-bbfe-4519-b239-157fa4382b21") : secret "image-registry-tls" not found
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.888221    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.888398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/76f6fe48-c826-4af3-bc3d-ab70010dcb93-snapshots\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.888186    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f6fe48-c826-4af3-bc3d-ab70010dcb93-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.888460    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls podName:92430eb1-2164-4d31-90dc-d80938eaacb7 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:18.388443427 +0000 UTC m=+35.649418606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-947dp" (UID: "92430eb1-2164-4d31-90dc-d80938eaacb7") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:17.888709 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.888671    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33096a88-bbfe-4519-b239-157fa4382b21-ca-trust-extracted\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.889327 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.889242    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-trusted-ca\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.891180 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.891158    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/76f6fe48-c826-4af3-bc3d-ab70010dcb93-serving-cert\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.891278 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.891225    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-installation-pull-secrets\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.891278 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.891180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb075801-f23d-4b07-a3e8-35b0dc2bda86-serving-cert\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.891278 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.891203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-image-registry-private-configuration\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.897368 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.897181    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-bound-sa-token\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.897368 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.897313    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5tf4\" (UniqueName: \"kubernetes.io/projected/fb075801-f23d-4b07-a3e8-35b0dc2bda86-kube-api-access-r5tf4\") pod \"console-operator-584dcb6b54-xjqb6\" (UID: \"fb075801-f23d-4b07-a3e8-35b0dc2bda86\") " pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:17.897520 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.897424    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fphqz\" (UniqueName: \"kubernetes.io/projected/5410afce-d2a3-4357-9959-fd06d74ec212-kube-api-access-fphqz\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:17.897654 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.897611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4lgq\" (UniqueName: \"kubernetes.io/projected/76f6fe48-c826-4af3-bc3d-ab70010dcb93-kube-api-access-s4lgq\") pod \"insights-operator-5885fdb759-mqz4z\" (UID: \"76f6fe48-c826-4af3-bc3d-ab70010dcb93\") " pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.899659 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.899596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvmh7\" (UniqueName: \"kubernetes.io/projected/92430eb1-2164-4d31-90dc-d80938eaacb7-kube-api-access-gvmh7\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:17.900020 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.900005    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdsb7\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-kube-api-access-kdsb7\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:17.933661 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.933637    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhdzk_60f24a69-e287-4fc4-a659-e0d275e17215/konnectivity-agent/0.log" limit=1024
Jul 27 17:21:17.949535 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.949513    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-mqz4z"
Jul 27 17:21:17.987837 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.987811    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-stats-auth\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.988022 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.987853    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2529\" (UniqueName: \"kubernetes.io/projected/668f942d-f6dc-4152-9bd0-870aa27229ba-kube-api-access-q2529\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.988022 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.987903    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.988022 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.987929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/668f942d-f6dc-4152-9bd0-870aa27229ba-config\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.988022 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.987948    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghqdj\" (UniqueName: \"kubernetes.io/projected/29645a34-e258-423f-acbd-aa709a57ffd7-kube-api-access-ghqdj\") pod \"ingress-canary-7r696\" (UID: \"29645a34-e258-423f-acbd-aa709a57ffd7\") " pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:17.988398 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/668f942d-f6dc-4152-9bd0-870aa27229ba-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.988398 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29645a34-e258-423f-acbd-aa709a57ffd7-cert\") pod \"ingress-canary-7r696\" (UID: \"29645a34-e258-423f-acbd-aa709a57ffd7\") " pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:17.988398 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988217    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.988398 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76cab81e-4e12-45f8-b53c-2134434c8663-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:17.988664 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988604    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/668f942d-f6dc-4152-9bd0-870aa27229ba-config\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.988720 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988688    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.988720 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.988697    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:21:17.988823 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.988775    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:18.488755228 +0000 UTC m=+35.749730618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : secret "router-metrics-certs-default" not found
Jul 27 17:21:17.988823 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988799    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.988988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988841    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lc7v\" (UniqueName: \"kubernetes.io/projected/9d428abe-51d0-42d3-b610-0552dd60f476-kube-api-access-4lc7v\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.988988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kxq\" (UniqueName: \"kubernetes.io/projected/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-kube-api-access-k2kxq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.988988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988895    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b428v\" (UniqueName: \"kubernetes.io/projected/123a82b2-e990-49bd-897a-4628586d8e49-kube-api-access-b428v\") pod \"volume-data-source-validator-549c6d54cf-fmm85\" (UID: \"123a82b2-e990-49bd-897a-4628586d8e49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"
Jul 27 17:21:17.988988 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.988910    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:18.488898316 +0000 UTC m=+35.749873498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : configmap references non-existent config key: service-ca.crt
Jul 27 17:21:17.988988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.988978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989009    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ccqr\" (UniqueName: \"kubernetes.io/projected/ee9b370c-7951-4a22-89bc-596ef49a2ad4-kube-api-access-4ccqr\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-default-certificate\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989062    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/76cab81e-4e12-45f8-b53c-2134434c8663-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tdd7\" (UniqueName: \"kubernetes.io/projected/3ffca930-1a03-46d2-9277-48017eeb1a11-kube-api-access-9tdd7\") pod \"network-check-source-7d88cb55d8-wqszm\" (UID: \"3ffca930-1a03-46d2-9277-48017eeb1a11\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9b370c-7951-4a22-89bc-596ef49a2ad4-config-volume\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989134    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9b370c-7951-4a22-89bc-596ef49a2ad4-tmp-dir\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.989103    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.989220    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert podName:76cab81e-4e12-45f8-b53c-2134434c8663 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:18.489203481 +0000 UTC m=+35.750178664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pgb6x" (UID: "76cab81e-4e12-45f8-b53c-2134434c8663") : secret "networking-console-plugin-cert" not found
Jul 27 17:21:17.989251 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.989256    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:21:17.989802 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:17.989310    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls podName:ee9b370c-7951-4a22-89bc-596ef49a2ad4 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:18.489279429 +0000 UTC m=+35.750254620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls") pod "dns-default-xxzzn" (UID: "ee9b370c-7951-4a22-89bc-596ef49a2ad4") : secret "dns-default-metrics-tls" not found
Jul 27 17:21:17.989802 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9b370c-7951-4a22-89bc-596ef49a2ad4-tmp-dir\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.989802 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.989686    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9b370c-7951-4a22-89bc-596ef49a2ad4-config-volume\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:17.991862 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.991833    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29645a34-e258-423f-acbd-aa709a57ffd7-cert\") pod \"ingress-canary-7r696\" (UID: \"29645a34-e258-423f-acbd-aa709a57ffd7\") " pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:17.992056 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.992040    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/668f942d-f6dc-4152-9bd0-870aa27229ba-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:17.992171 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.992097    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:17.993659 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.993590    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-default-certificate\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:17.994051 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:17.994031    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-stats-auth\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:18.001668 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.001642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:18.002590 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.002567    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2529\" (UniqueName: \"kubernetes.io/projected/668f942d-f6dc-4152-9bd0-870aa27229ba-kube-api-access-q2529\") pod \"service-ca-operator-5d6c75f84f-bhxjj\" (UID: \"668f942d-f6dc-4152-9bd0-870aa27229ba\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:18.003157 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.003093    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ccqr\" (UniqueName: \"kubernetes.io/projected/ee9b370c-7951-4a22-89bc-596ef49a2ad4-kube-api-access-4ccqr\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:18.003683 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.003657    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2kxq\" (UniqueName: \"kubernetes.io/projected/1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be-kube-api-access-k2kxq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9\" (UID: \"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:18.006310 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.006037    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghqdj\" (UniqueName: \"kubernetes.io/projected/29645a34-e258-423f-acbd-aa709a57ffd7-kube-api-access-ghqdj\") pod \"ingress-canary-7r696\" (UID: \"29645a34-e258-423f-acbd-aa709a57ffd7\") " pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:18.006310 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.006252    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lc7v\" (UniqueName: \"kubernetes.io/projected/9d428abe-51d0-42d3-b610-0552dd60f476-kube-api-access-4lc7v\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:18.006857 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.006717    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tdd7\" (UniqueName: \"kubernetes.io/projected/3ffca930-1a03-46d2-9277-48017eeb1a11-kube-api-access-9tdd7\") pod \"network-check-source-7d88cb55d8-wqszm\" (UID: \"3ffca930-1a03-46d2-9277-48017eeb1a11\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"
Jul 27 17:21:18.007214 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.007195    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b428v\" (UniqueName: \"kubernetes.io/projected/123a82b2-e990-49bd-897a-4628586d8e49-kube-api-access-b428v\") pod \"volume-data-source-validator-549c6d54cf-fmm85\" (UID: \"123a82b2-e990-49bd-897a-4628586d8e49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"
Jul 27 17:21:18.019036 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.018906    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"
Jul 27 17:21:18.046595 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.046201    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"
Jul 27 17:21:18.064030 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.063367    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"
Jul 27 17:21:18.096580 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.096371    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"
Jul 27 17:21:18.112641 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.112579    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7r696"
Jul 27 17:21:18.162186 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.162166    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mqz4z"]
Jul 27 17:21:18.169258 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:18.168945    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f6fe48_c826_4af3_bc3d_ab70010dcb93.slice/crio-d418d97adb4d300e4fd3236705a136416b7108c5d2850df81bca550718afd5b7 WatchSource:0}: Error finding container d418d97adb4d300e4fd3236705a136416b7108c5d2850df81bca550718afd5b7: Status 404 returned error can't find the container with id d418d97adb4d300e4fd3236705a136416b7108c5d2850df81bca550718afd5b7
Jul 27 17:21:18.228088 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.228015    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xjqb6"]
Jul 27 17:21:18.250082 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.249682    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:18.250189 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.250107    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:18.258342 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.253200    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:21:18.258342 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.253455    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfvk\""
Jul 27 17:21:18.258342 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.253490    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:21:18.276989 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.276779    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85"]
Jul 27 17:21:18.282726 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:18.282696    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123a82b2_e990_49bd_897a_4628586d8e49.slice/crio-78fc7eb41fe899a19db5aed2939afc37fabd19b1d778e2f5ab78baafed52bb45 WatchSource:0}: Error finding container 78fc7eb41fe899a19db5aed2939afc37fabd19b1d778e2f5ab78baafed52bb45: Status 404 returned error can't find the container with id 78fc7eb41fe899a19db5aed2939afc37fabd19b1d778e2f5ab78baafed52bb45
Jul 27 17:21:18.289032 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.289008    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj"]
Jul 27 17:21:18.294632 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:18.294475    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668f942d_f6dc_4152_9bd0_870aa27229ba.slice/crio-824674d7d4b919c7feed4946dfc5a2568e499c5237e7d641102d76d12d9778ba WatchSource:0}: Error finding container 824674d7d4b919c7feed4946dfc5a2568e499c5237e7d641102d76d12d9778ba: Status 404 returned error can't find the container with id 824674d7d4b919c7feed4946dfc5a2568e499c5237e7d641102d76d12d9778ba
Jul 27 17:21:18.313883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.313859    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9"]
Jul 27 17:21:18.316662 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:18.316634    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce5495a_aaf3_4fd5_81cd_4ba5751dd6be.slice/crio-665360e4fbdac6d250ff6bea7eb0f1933dbf8fb894f1b9108525014d50493f31 WatchSource:0}: Error finding container 665360e4fbdac6d250ff6bea7eb0f1933dbf8fb894f1b9108525014d50493f31: Status 404 returned error can't find the container with id 665360e4fbdac6d250ff6bea7eb0f1933dbf8fb894f1b9108525014d50493f31
Jul 27 17:21:18.325130 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.325015    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm"]
Jul 27 17:21:18.327549 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:18.327528    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffca930_1a03_46d2_9277_48017eeb1a11.slice/crio-dde8f194a0635d3c815c62d73e1a52cab1be62c5c2bcdf8e59003b0055fe1174 WatchSource:0}: Error finding container dde8f194a0635d3c815c62d73e1a52cab1be62c5c2bcdf8e59003b0055fe1174: Status 404 returned error can't find the container with id dde8f194a0635d3c815c62d73e1a52cab1be62c5c2bcdf8e59003b0055fe1174
Jul 27 17:21:18.344258 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.344177    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7r696"]
Jul 27 17:21:18.346879 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:18.346851    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29645a34_e258_423f_acbd_aa709a57ffd7.slice/crio-25285a1e0887d031bcfb384cf4619663bbdba1f0f58c7e9607a3a1703246fdee WatchSource:0}: Error finding container 25285a1e0887d031bcfb384cf4619663bbdba1f0f58c7e9607a3a1703246fdee: Status 404 returned error can't find the container with id 25285a1e0887d031bcfb384cf4619663bbdba1f0f58c7e9607a3a1703246fdee
Jul 27 17:21:18.394095 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.394062    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:18.394210 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.394134    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:18.394210 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.394163    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:18.394355 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.394336    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:18.394470 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.394399    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls podName:92430eb1-2164-4d31-90dc-d80938eaacb7 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:19.394380272 +0000 UTC m=+36.655355453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-947dp" (UID: "92430eb1-2164-4d31-90dc-d80938eaacb7") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:18.394470 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.394429    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:21:18.394600 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.394487    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls podName:5410afce-d2a3-4357-9959-fd06d74ec212 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:19.394470706 +0000 UTC m=+36.655445902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls") pod "cluster-samples-operator-6748489656-l8qgt" (UID: "5410afce-d2a3-4357-9959-fd06d74ec212") : secret "samples-operator-tls" not found
Jul 27 17:21:18.394600 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.394428    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:21:18.394600 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.394508    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86fbf99f7-pzptv: secret "image-registry-tls" not found
Jul 27 17:21:18.394600 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.394541    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls podName:33096a88-bbfe-4519-b239-157fa4382b21 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:19.394530221 +0000 UTC m=+36.655505398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls") pod "image-registry-86fbf99f7-pzptv" (UID: "33096a88-bbfe-4519-b239-157fa4382b21") : secret "image-registry-tls" not found
Jul 27 17:21:18.423614 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.423589    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7r696" event={"ID":"29645a34-e258-423f-acbd-aa709a57ffd7","Type":"ContainerStarted","Data":"25285a1e0887d031bcfb384cf4619663bbdba1f0f58c7e9607a3a1703246fdee"}
Jul 27 17:21:18.425055 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.425027    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9" event={"ID":"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be","Type":"ContainerStarted","Data":"665360e4fbdac6d250ff6bea7eb0f1933dbf8fb894f1b9108525014d50493f31"}
Jul 27 17:21:18.426245 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.426224    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj" event={"ID":"668f942d-f6dc-4152-9bd0-870aa27229ba","Type":"ContainerStarted","Data":"824674d7d4b919c7feed4946dfc5a2568e499c5237e7d641102d76d12d9778ba"}
Jul 27 17:21:18.427567 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.427544    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm" event={"ID":"3ffca930-1a03-46d2-9277-48017eeb1a11","Type":"ContainerStarted","Data":"dde8f194a0635d3c815c62d73e1a52cab1be62c5c2bcdf8e59003b0055fe1174"}
Jul 27 17:21:18.428844 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.428825    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mqz4z" event={"ID":"76f6fe48-c826-4af3-bc3d-ab70010dcb93","Type":"ContainerStarted","Data":"d418d97adb4d300e4fd3236705a136416b7108c5d2850df81bca550718afd5b7"}
Jul 27 17:21:18.430090 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.430066    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85" event={"ID":"123a82b2-e990-49bd-897a-4628586d8e49","Type":"ContainerStarted","Data":"78fc7eb41fe899a19db5aed2939afc37fabd19b1d778e2f5ab78baafed52bb45"}
Jul 27 17:21:18.431260 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.431239    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" event={"ID":"fb075801-f23d-4b07-a3e8-35b0dc2bda86","Type":"ContainerStarted","Data":"c6b05f65df98748f374b9608b0d9ee2c8c379aa2c4968cbc73e28f1688750a12"}
Jul 27 17:21:18.494795 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.494769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:18.494925 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.494820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:18.494925 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.494873    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:18.495040 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:18.494934    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:18.495040 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.494935    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:21:18.495040 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.495026    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:19.495007274 +0000 UTC m=+36.755982457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : configmap references non-existent config key: service-ca.crt
Jul 27 17:21:18.495190 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.495056    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:19.49504012 +0000 UTC m=+36.756015314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : secret "router-metrics-certs-default" not found
Jul 27 17:21:18.495190 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.495129    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 17:21:18.495190 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.495165    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert podName:76cab81e-4e12-45f8-b53c-2134434c8663 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:19.495153134 +0000 UTC m=+36.756128315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pgb6x" (UID: "76cab81e-4e12-45f8-b53c-2134434c8663") : secret "networking-console-plugin-cert" not found
Jul 27 17:21:18.495401 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.495221    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:21:18.495401 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:18.495277    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls podName:ee9b370c-7951-4a22-89bc-596ef49a2ad4 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:19.495251226 +0000 UTC m=+36.756226406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls") pod "dns-default-xxzzn" (UID: "ee9b370c-7951-4a22-89bc-596ef49a2ad4") : secret "dns-default-metrics-tls" not found
Jul 27 17:21:19.250311 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.250260    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:19.253956 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.253931    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dgczs\""
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.404187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.404263    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.404316    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.404555    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.404619    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls podName:92430eb1-2164-4d31-90dc-d80938eaacb7 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:21.404599837 +0000 UTC m=+38.665575018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-947dp" (UID: "92430eb1-2164-4d31-90dc-d80938eaacb7") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.405055    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.405069    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86fbf99f7-pzptv: secret "image-registry-tls" not found
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.405113    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls podName:33096a88-bbfe-4519-b239-157fa4382b21 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:21.405097447 +0000 UTC m=+38.666072627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls") pod "image-registry-86fbf99f7-pzptv" (UID: "33096a88-bbfe-4519-b239-157fa4382b21") : secret "image-registry-tls" not found
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.405173    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:21:19.405242 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.405203    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls podName:5410afce-d2a3-4357-9959-fd06d74ec212 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:21.40519327 +0000 UTC m=+38.666168461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls") pod "cluster-samples-operator-6748489656-l8qgt" (UID: "5410afce-d2a3-4357-9959-fd06d74ec212") : secret "samples-operator-tls" not found
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.505578    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.505647    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.505716    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.505779    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:21.50575714 +0000 UTC m=+38.766732318 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : configmap references non-existent config key: service-ca.crt
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.505860    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.505871    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:19.505890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.505908    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls podName:ee9b370c-7951-4a22-89bc-596ef49a2ad4 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:21.505892714 +0000 UTC m=+38.766867895 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls") pod "dns-default-xxzzn" (UID: "ee9b370c-7951-4a22-89bc-596ef49a2ad4") : secret "dns-default-metrics-tls" not found
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.505926    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert podName:76cab81e-4e12-45f8-b53c-2134434c8663 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:21.505916013 +0000 UTC m=+38.766891193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pgb6x" (UID: "76cab81e-4e12-45f8-b53c-2134434c8663") : secret "networking-console-plugin-cert" not found
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.505960    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:21:19.506338 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:19.506005    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:21.505991551 +0000 UTC m=+38.766966732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : secret "router-metrics-certs-default" not found
Jul 27 17:21:21.425963 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:21.425923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:21.426004    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:21.426034    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.426237    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.426320    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls podName:92430eb1-2164-4d31-90dc-d80938eaacb7 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:25.42628221 +0000 UTC m=+42.687257390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-947dp" (UID: "92430eb1-2164-4d31-90dc-d80938eaacb7") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.426333    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.426345    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86fbf99f7-pzptv: secret "image-registry-tls" not found
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.426380    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls podName:33096a88-bbfe-4519-b239-157fa4382b21 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:25.426369261 +0000 UTC m=+42.687344452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls") pod "image-registry-86fbf99f7-pzptv" (UID: "33096a88-bbfe-4519-b239-157fa4382b21") : secret "image-registry-tls" not found
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.426436    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:21:21.426546 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.426466    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls podName:5410afce-d2a3-4357-9959-fd06d74ec212 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:25.426455077 +0000 UTC m=+42.687430257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls") pod "cluster-samples-operator-6748489656-l8qgt" (UID: "5410afce-d2a3-4357-9959-fd06d74ec212") : secret "samples-operator-tls" not found
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:21.528950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:21.529096    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:21.529129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:21.529176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.529329    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.529397    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert podName:76cab81e-4e12-45f8-b53c-2134434c8663 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:25.52937718 +0000 UTC m=+42.790352360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pgb6x" (UID: "76cab81e-4e12-45f8-b53c-2134434c8663") : secret "networking-console-plugin-cert" not found
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.529831    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.529884    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls podName:ee9b370c-7951-4a22-89bc-596ef49a2ad4 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:25.529867782 +0000 UTC m=+42.790842963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls") pod "dns-default-xxzzn" (UID: "ee9b370c-7951-4a22-89bc-596ef49a2ad4") : secret "dns-default-metrics-tls" not found
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.529947    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.529980    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:25.529969301 +0000 UTC m=+42.790944480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : secret "router-metrics-certs-default" not found
Jul 27 17:21:21.530105 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:21.530047    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:25.530037855 +0000 UTC m=+42.791013036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : configmap references non-existent config key: service-ca.crt
Jul 27 17:21:25.465966 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:25.465931    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:25.465996    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:25.466024    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.466096    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.466114    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86fbf99f7-pzptv: secret "image-registry-tls" not found
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.466119    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.466166    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.466179    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls podName:33096a88-bbfe-4519-b239-157fa4382b21 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.466158523 +0000 UTC m=+50.727133724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls") pod "image-registry-86fbf99f7-pzptv" (UID: "33096a88-bbfe-4519-b239-157fa4382b21") : secret "image-registry-tls" not found
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.466227    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls podName:5410afce-d2a3-4357-9959-fd06d74ec212 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.466206902 +0000 UTC m=+50.727182104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls") pod "cluster-samples-operator-6748489656-l8qgt" (UID: "5410afce-d2a3-4357-9959-fd06d74ec212") : secret "samples-operator-tls" not found
Jul 27 17:21:25.466414 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.466245    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls podName:92430eb1-2164-4d31-90dc-d80938eaacb7 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.466238733 +0000 UTC m=+50.727213909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-947dp" (UID: "92430eb1-2164-4d31-90dc-d80938eaacb7") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:25.566468 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:25.566434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:25.566646 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:25.566538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:25.566646 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:25.566565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:25.566646 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.566597    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:21:25.566646 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:25.566613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:25.566868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.566656    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls podName:ee9b370c-7951-4a22-89bc-596ef49a2ad4 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.566641071 +0000 UTC m=+50.827616249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls") pod "dns-default-xxzzn" (UID: "ee9b370c-7951-4a22-89bc-596ef49a2ad4") : secret "dns-default-metrics-tls" not found
Jul 27 17:21:25.566868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.566697    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 17:21:25.566868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.566711    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.566694892 +0000 UTC m=+50.827670085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : configmap references non-existent config key: service-ca.crt
Jul 27 17:21:25.566868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.566698    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:21:25.566868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.566735    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert podName:76cab81e-4e12-45f8-b53c-2134434c8663 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.56672373 +0000 UTC m=+50.827698908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pgb6x" (UID: "76cab81e-4e12-45f8-b53c-2134434c8663") : secret "networking-console-plugin-cert" not found
Jul 27 17:21:25.566868 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:25.566750    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.566741065 +0000 UTC m=+50.827716243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : secret "router-metrics-certs-default" not found
Jul 27 17:21:29.466276 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.466238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj" event={"ID":"668f942d-f6dc-4152-9bd0-870aa27229ba","Type":"ContainerStarted","Data":"d23c80deb5ca58b697019d6689240862403133b09cf636cc827ea306049145bf"}
Jul 27 17:21:29.467840 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.467815    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm" event={"ID":"3ffca930-1a03-46d2-9277-48017eeb1a11","Type":"ContainerStarted","Data":"f26ad74a05d5371e93f87e1e9def0f98aa9596bc20c837fdb5b9ce4ae6e863b9"}
Jul 27 17:21:29.469345 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.469323    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mqz4z" event={"ID":"76f6fe48-c826-4af3-bc3d-ab70010dcb93","Type":"ContainerStarted","Data":"a3b7f68aee04a7d2f5a053b1ab717454cd74c3bdc30a02d34a7230c79227fb0e"}
Jul 27 17:21:29.472480 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.472455    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b483d54-7e05-4952-82b2-e25106bd4549" containerID="2ce5f6e3272d87fdd49ce3265d31ff2411558fa2f721976768b819e2fed2efe1" exitCode=0
Jul 27 17:21:29.472585 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.472534    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerDied","Data":"2ce5f6e3272d87fdd49ce3265d31ff2411558fa2f721976768b819e2fed2efe1"}
Jul 27 17:21:29.476709 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.476690    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85" event={"ID":"123a82b2-e990-49bd-897a-4628586d8e49","Type":"ContainerStarted","Data":"98b0b51bc73196d2f376ce4bea61392a53a2c4e867cdc8fad86fff91ae7f6469"}
Jul 27 17:21:29.478497 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.478478    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/0.log"
Jul 27 17:21:29.478591 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.478514    2565 generic.go:356] "Generic (PLEG): container finished" podID="fb075801-f23d-4b07-a3e8-35b0dc2bda86" containerID="75919e99c3d911474e6fed07e53449f0207911c24a818855365c1cb2b5bdaa08" exitCode=255
Jul 27 17:21:29.478591 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.478585    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" event={"ID":"fb075801-f23d-4b07-a3e8-35b0dc2bda86","Type":"ContainerDied","Data":"75919e99c3d911474e6fed07e53449f0207911c24a818855365c1cb2b5bdaa08"}
Jul 27 17:21:29.478871 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.478853    2565 scope.go:117] "RemoveContainer" containerID="75919e99c3d911474e6fed07e53449f0207911c24a818855365c1cb2b5bdaa08"
Jul 27 17:21:29.482343 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.482322    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7r696" event={"ID":"29645a34-e258-423f-acbd-aa709a57ffd7","Type":"ContainerStarted","Data":"6a7ba70f08c92059edb78e9c0ee35049cb38c40e69fe38d8a192f282127403d4"}
Jul 27 17:21:29.485591 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.484433    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj" podStartSLOduration=26.14499812 podStartE2EDuration="36.484418954s" totalImagesPullingTime="10.339420834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:29.483689762 +0000 UTC m=+46.744664959" watchObservedRunningTime="2026-07-27 17:21:29.484418954 +0000 UTC m=+46.745394151"
Jul 27 17:21:29.486008 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.485977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9" event={"ID":"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be","Type":"ContainerStarted","Data":"eeeec3086b8e34a9028561df384c48562179ea0cb1396bbb2f28eeee8f57266c"}
Jul 27 17:21:29.552818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.552766    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-wqszm" podStartSLOduration=26.245835774 podStartE2EDuration="36.552745439s" totalImagesPullingTime="10.306909665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:29.520871511 +0000 UTC m=+46.781846710" watchObservedRunningTime="2026-07-27 17:21:29.552745439 +0000 UTC m=+46.813720639"
Jul 27 17:21:29.570556 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.570432    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-mqz4z" podStartSLOduration=26.106651723 podStartE2EDuration="36.570413378s" totalImagesPullingTime="10.463761655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:29.569909945 +0000 UTC m=+46.830885145" watchObservedRunningTime="2026-07-27 17:21:29.570413378 +0000 UTC m=+46.831388576"
Jul 27 17:21:29.606276 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.604710    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fmm85" podStartSLOduration=26.254157759 podStartE2EDuration="36.604690459s" totalImagesPullingTime="10.3505327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:29.589390535 +0000 UTC m=+46.850365735" watchObservedRunningTime="2026-07-27 17:21:29.604690459 +0000 UTC m=+46.865665660"
Jul 27 17:21:29.606276 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.605995    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7r696" podStartSLOduration=2.312245718 podStartE2EDuration="12.605978517s" totalImagesPullingTime="10.293732799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:29.605356039 +0000 UTC m=+46.866331239" watchObservedRunningTime="2026-07-27 17:21:29.605978517 +0000 UTC m=+46.866953717"
Jul 27 17:21:29.625036 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.624226    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9" podStartSLOduration=26.308530092 podStartE2EDuration="36.624206833s" totalImagesPullingTime="10.315676741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:29.623202169 +0000 UTC m=+46.884177369" watchObservedRunningTime="2026-07-27 17:21:29.624206833 +0000 UTC m=+46.885182033"
Jul 27 17:21:29.933430 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.932043    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8zj7b"]
Jul 27 17:21:29.951863 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.951823    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:29.952675 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.952650    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8zj7b"]
Jul 27 17:21:29.955560 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.955529    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 17:21:29.955718 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.955625    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 17:21:29.955976 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:29.955946    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bfsxf\""
Jul 27 17:21:30.018327 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.018240    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-crio-socket\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.018515 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.018497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-data-volume\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.018644 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.018630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpzsb\" (UniqueName: \"kubernetes.io/projected/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-api-access-cpzsb\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.018861 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.018846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.019114 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.019098    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.120371 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.120336    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.120549 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.120486    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:21:30.120634 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.120553    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.120634 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.120568    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:30.620546798 +0000 UTC m=+47.881521987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : configmap "kube-rbac-proxy" not found
Jul 27 17:21:30.120634 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.120627    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-crio-socket\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.120802 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.120652    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:30.120802 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.120660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-data-volume\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.120802 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.120689    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpzsb\" (UniqueName: \"kubernetes.io/projected/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-api-access-cpzsb\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.120802 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.120703    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:30.620687164 +0000 UTC m=+47.881662359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:30.120802 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.120776    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-crio-socket\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.121128 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.121107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-data-volume\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.130721 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.130688    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpzsb\" (UniqueName: \"kubernetes.io/projected/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-api-access-cpzsb\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.493135 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.493097    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b483d54-7e05-4952-82b2-e25106bd4549" containerID="8d2a831c1c732fcd91fcb84cf8085419cc51b0cc8012a02f7abd93c63cdb18f4" exitCode=0
Jul 27 17:21:30.493609 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.493179    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerDied","Data":"8d2a831c1c732fcd91fcb84cf8085419cc51b0cc8012a02f7abd93c63cdb18f4"}
Jul 27 17:21:30.495001 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.494975    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:21:30.495476 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.495456    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/0.log"
Jul 27 17:21:30.495547 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.495495    2565 generic.go:356] "Generic (PLEG): container finished" podID="fb075801-f23d-4b07-a3e8-35b0dc2bda86" containerID="b0e8408cd92b20ee515e5892391dd75190dba69f15fedfd2601633991a58b143" exitCode=255
Jul 27 17:21:30.495663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.495637    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" event={"ID":"fb075801-f23d-4b07-a3e8-35b0dc2bda86","Type":"ContainerDied","Data":"b0e8408cd92b20ee515e5892391dd75190dba69f15fedfd2601633991a58b143"}
Jul 27 17:21:30.495728 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.495697    2565 scope.go:117] "RemoveContainer" containerID="75919e99c3d911474e6fed07e53449f0207911c24a818855365c1cb2b5bdaa08"
Jul 27 17:21:30.495780 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.495763    2565 scope.go:117] "RemoveContainer" containerID="b0e8408cd92b20ee515e5892391dd75190dba69f15fedfd2601633991a58b143"
Jul 27 17:21:30.495967 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.495945    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xjqb6_openshift-console-operator(fb075801-f23d-4b07-a3e8-35b0dc2bda86)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" podUID="fb075801-f23d-4b07-a3e8-35b0dc2bda86"
Jul 27 17:21:30.628753 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.628718    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.628883 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.628861    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:30.629157 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.629014    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:21:30.629157 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.629091    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:31.629072763 +0000 UTC m=+48.890047944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : configmap "kube-rbac-proxy" not found
Jul 27 17:21:30.629157 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.629151    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:30.629424 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:30.629193    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:31.629180837 +0000 UTC m=+48.890156027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:30.929441 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:30.929413    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2kb9q_4f91c41b-a2d4-4635-840f-09bf2302fd91/dns-node-resolver/0.log"
Jul 27 17:21:31.502460 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.502427    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" event={"ID":"6b483d54-7e05-4952-82b2-e25106bd4549","Type":"ContainerStarted","Data":"32e6eea5231de21f82621ba2031ac7645e597e1e0fd6411e0bbd7bdb6ab5a7cb"}
Jul 27 17:21:31.503869 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.503850    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:21:31.504118 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.504104    2565 scope.go:117] "RemoveContainer" containerID="b0e8408cd92b20ee515e5892391dd75190dba69f15fedfd2601633991a58b143"
Jul 27 17:21:31.504247 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:31.504231    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xjqb6_openshift-console-operator(fb075801-f23d-4b07-a3e8-35b0dc2bda86)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" podUID="fb075801-f23d-4b07-a3e8-35b0dc2bda86"
Jul 27 17:21:31.528247 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.528203    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nfjn2" podStartSLOduration=9.564610474 podStartE2EDuration="48.528189856s" totalImagesPullingTime="38.963579382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:31.526445264 +0000 UTC m=+48.787420466" watchObservedRunningTime="2026-07-27 17:21:31.528189856 +0000 UTC m=+48.789165055"
Jul 27 17:21:31.638632 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.638590    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:31.638785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.638696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:31.638785 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:31.638708    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:21:31.638785 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:31.638769    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.63875235 +0000 UTC m=+50.899727531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : configmap "kube-rbac-proxy" not found
Jul 27 17:21:31.638932 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:31.638794    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:31.638932 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:31.638839    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:33.638819126 +0000 UTC m=+50.899794304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:31.696555 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.696524    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"]
Jul 27 17:21:31.721029 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.721000    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"]
Jul 27 17:21:31.721172 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.721096    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"
Jul 27 17:21:31.723805 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.723781    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 27 17:21:31.723805 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.723793    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 27 17:21:31.723964 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.723783    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hgdfc\""
Jul 27 17:21:31.840795 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.840761    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g9fs\" (UniqueName: \"kubernetes.io/projected/7c73813b-830d-4f2a-8163-49d6a971270a-kube-api-access-8g9fs\") pod \"migrator-6865c9967-5wsx6\" (UID: \"7c73813b-830d-4f2a-8163-49d6a971270a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"
Jul 27 17:21:31.929513 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.929485    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c7rx4_bfe4dd64-9586-428a-a704-c3d5cec3b3e7/node-ca/0.log"
Jul 27 17:21:31.941811 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.941784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g9fs\" (UniqueName: \"kubernetes.io/projected/7c73813b-830d-4f2a-8163-49d6a971270a-kube-api-access-8g9fs\") pod \"migrator-6865c9967-5wsx6\" (UID: \"7c73813b-830d-4f2a-8163-49d6a971270a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"
Jul 27 17:21:31.950982 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:31.950963    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g9fs\" (UniqueName: \"kubernetes.io/projected/7c73813b-830d-4f2a-8163-49d6a971270a-kube-api-access-8g9fs\") pod \"migrator-6865c9967-5wsx6\" (UID: \"7c73813b-830d-4f2a-8163-49d6a971270a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"
Jul 27 17:21:32.029979 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:32.029953    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"
Jul 27 17:21:32.153207 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:32.153176    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6"]
Jul 27 17:21:32.156365 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:32.156328    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c73813b_830d_4f2a_8163_49d6a971270a.slice/crio-2f8eab7a2cf6d26f8fa887a6f4535717948c36092cb73c43c9bdb4863659c053 WatchSource:0}: Error finding container 2f8eab7a2cf6d26f8fa887a6f4535717948c36092cb73c43c9bdb4863659c053: Status 404 returned error can't find the container with id 2f8eab7a2cf6d26f8fa887a6f4535717948c36092cb73c43c9bdb4863659c053
Jul 27 17:21:32.510096 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:32.510065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6" event={"ID":"7c73813b-830d-4f2a-8163-49d6a971270a","Type":"ContainerStarted","Data":"2f8eab7a2cf6d26f8fa887a6f4535717948c36092cb73c43c9bdb4863659c053"}
Jul 27 17:21:32.732464 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:32.732385    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7r696_29645a34-e258-423f-acbd-aa709a57ffd7/serve-healthcheck-canary/0.log"
Jul 27 17:21:33.556988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.556954    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.557014    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.557098    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.557117    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.557137    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86fbf99f7-pzptv: secret "image-registry-tls" not found
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.557143    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls podName:5410afce-d2a3-4357-9959-fd06d74ec212 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:49.557129466 +0000 UTC m=+66.818104643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls") pod "cluster-samples-operator-6748489656-l8qgt" (UID: "5410afce-d2a3-4357-9959-fd06d74ec212") : secret "samples-operator-tls" not found
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.557177    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.557187    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls podName:33096a88-bbfe-4519-b239-157fa4382b21 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:49.557170946 +0000 UTC m=+66.818146143 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls") pod "image-registry-86fbf99f7-pzptv" (UID: "33096a88-bbfe-4519-b239-157fa4382b21") : secret "image-registry-tls" not found
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.557252    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:33.557384 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.557360    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls podName:92430eb1-2164-4d31-90dc-d80938eaacb7 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:49.557288639 +0000 UTC m=+66.818263817 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-947dp" (UID: "92430eb1-2164-4d31-90dc-d80938eaacb7") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:21:33.657895 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.657870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:33.657976 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.657905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:33.657976 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.657941    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:33.658081 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.657988    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:33.658081 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658012    2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 17:21:33.658081 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.658032    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:33.658081 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658064    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert podName:76cab81e-4e12-45f8-b53c-2134434c8663 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:49.658050099 +0000 UTC m=+66.919025276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-pgb6x" (UID: "76cab81e-4e12-45f8-b53c-2134434c8663") : secret "networking-console-plugin-cert" not found
Jul 27 17:21:33.658253 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:33.658083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:33.658253 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658014    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:33.658253 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658087    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:21:33.658253 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658161    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:49.658150359 +0000 UTC m=+66.919125540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : configmap references non-existent config key: service-ca.crt
Jul 27 17:21:33.658253 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658106    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:21:33.658253 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658112    2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:21:33.658253 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658183    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:37.658173303 +0000 UTC m=+54.919148482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:33.658511 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658268    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:37.658250251 +0000 UTC m=+54.919225432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : configmap "kube-rbac-proxy" not found
Jul 27 17:21:33.658511 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658284    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls podName:ee9b370c-7951-4a22-89bc-596ef49a2ad4 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:49.65827739 +0000 UTC m=+66.919252570 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls") pod "dns-default-xxzzn" (UID: "ee9b370c-7951-4a22-89bc-596ef49a2ad4") : secret "dns-default-metrics-tls" not found
Jul 27 17:21:33.658511 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:33.658316    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs podName:9d428abe-51d0-42d3-b610-0552dd60f476 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:49.658310568 +0000 UTC m=+66.919285745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs") pod "router-default-58fd4bb687-9jltm" (UID: "9d428abe-51d0-42d3-b610-0552dd60f476") : secret "router-metrics-certs-default" not found
Jul 27 17:21:34.522930 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:34.522895    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6" event={"ID":"7c73813b-830d-4f2a-8163-49d6a971270a","Type":"ContainerStarted","Data":"10ef68b7643945268e80883054e158cb93342e795228f0074362c9027057956d"}
Jul 27 17:21:34.523111 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:34.522971    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6" event={"ID":"7c73813b-830d-4f2a-8163-49d6a971270a","Type":"ContainerStarted","Data":"60d70fcdc484253ad803fdeb8f5aa422a9fc8b51ffc57068cf1fda032a0ecd49"}
Jul 27 17:21:34.538541 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:34.538492    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-5wsx6" podStartSLOduration=2.13580255 podStartE2EDuration="3.538479442s" totalImagesPullingTime="1.402676892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:34.538383187 +0000 UTC m=+51.799358387" watchObservedRunningTime="2026-07-27 17:21:34.538479442 +0000 UTC m=+51.799454649"
Jul 27 17:21:37.695378 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:37.695326    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:37.695931 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:37.695444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:37.695931 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:37.695541    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:37.695931 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:37.695592    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls podName:830ccf32-fb7c-461b-bf99-4aa55b3ad9a5 nodeName:}" failed. No retries permitted until 2026-07-27 17:21:45.69558 +0000 UTC m=+62.956555177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8zj7b" (UID: "830ccf32-fb7c-461b-bf99-4aa55b3ad9a5") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:21:37.695931 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:37.695829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:38.002126 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:38.002029    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:38.002126 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:38.002096    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:38.002449 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:38.002437    2565 scope.go:117] "RemoveContainer" containerID="b0e8408cd92b20ee515e5892391dd75190dba69f15fedfd2601633991a58b143"
Jul 27 17:21:38.002592 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:21:38.002575    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xjqb6_openshift-console-operator(fb075801-f23d-4b07-a3e8-35b0dc2bda86)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" podUID="fb075801-f23d-4b07-a3e8-35b0dc2bda86"
Jul 27 17:21:42.412080 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:42.412050    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hv7q4"
Jul 27 17:21:45.772164 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:45.772129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:45.774507 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:45.774475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/830ccf32-fb7c-461b-bf99-4aa55b3ad9a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zj7b\" (UID: \"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5\") " pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:45.866527 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:45.866497    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bfsxf\""
Jul 27 17:21:45.873410 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:45.873387    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8zj7b"
Jul 27 17:21:46.003748 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:46.003724    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8zj7b"]
Jul 27 17:21:46.006202 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:46.006178    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830ccf32_fb7c_461b_bf99_4aa55b3ad9a5.slice/crio-4956c8d7cc415685c0594fad8a9e50a775d94c01f1d165f724e4d436cb90c1ee WatchSource:0}: Error finding container 4956c8d7cc415685c0594fad8a9e50a775d94c01f1d165f724e4d436cb90c1ee: Status 404 returned error can't find the container with id 4956c8d7cc415685c0594fad8a9e50a775d94c01f1d165f724e4d436cb90c1ee
Jul 27 17:21:46.565467 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:46.565389    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zj7b" event={"ID":"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5","Type":"ContainerStarted","Data":"4956c8d7cc415685c0594fad8a9e50a775d94c01f1d165f724e4d436cb90c1ee"}
Jul 27 17:21:48.899682 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:48.899641    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:48.903084 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:48.903065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:21:48.913115 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:48.913087    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4e307bf-7bb5-44dc-8dc0-890bbafffb78-original-pull-secret\") pod \"global-pull-secret-syncer-l288b\" (UID: \"e4e307bf-7bb5-44dc-8dc0-890bbafffb78\") " pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:49.000217 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.000183    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:49.000410 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.000258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:49.003019 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.002993    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96k4k\" (UniqueName: \"kubernetes.io/projected/c0ec967f-de07-46cc-899c-c66c49a81afd-kube-api-access-96k4k\") pod \"network-check-target-bdtq8\" (UID: \"c0ec967f-de07-46cc-899c-c66c49a81afd\") " pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:49.003568 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.003544    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:21:49.013108 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.013075    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12fa5ebb-f199-4903-8d8d-d571608e044b-metrics-certs\") pod \"network-metrics-daemon-dls6k\" (UID: \"12fa5ebb-f199-4903-8d8d-d571608e044b\") " pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:49.171475 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.171393    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l288b"
Jul 27 17:21:49.181588 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.181572    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfvk\""
Jul 27 17:21:49.189083 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.189059    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dls6k"
Jul 27 17:21:49.251079 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.251051    2565 scope.go:117] "RemoveContainer" containerID="b0e8408cd92b20ee515e5892391dd75190dba69f15fedfd2601633991a58b143"
Jul 27 17:21:49.269023 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.269002    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dgczs\""
Jul 27 17:21:49.276431 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.276410    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:49.606815 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.606784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:49.607032 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.606838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:49.607032 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.606870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:49.609537 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.609512    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"image-registry-86fbf99f7-pzptv\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") " pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:49.609839 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.609807    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5410afce-d2a3-4357-9959-fd06d74ec212-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-l8qgt\" (UID: \"5410afce-d2a3-4357-9959-fd06d74ec212\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:49.609929 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.609917    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/92430eb1-2164-4d31-90dc-d80938eaacb7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-947dp\" (UID: \"92430eb1-2164-4d31-90dc-d80938eaacb7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:49.708346 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.708306    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:49.708346 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.708354    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:49.708615 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.708485    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:49.708615 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.708554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:49.709488 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.709456    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d428abe-51d0-42d3-b610-0552dd60f476-service-ca-bundle\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:49.713710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.711331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d428abe-51d0-42d3-b610-0552dd60f476-metrics-certs\") pod \"router-default-58fd4bb687-9jltm\" (UID: \"9d428abe-51d0-42d3-b610-0552dd60f476\") " pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:49.713710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.711531    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9b370c-7951-4a22-89bc-596ef49a2ad4-metrics-tls\") pod \"dns-default-xxzzn\" (UID: \"ee9b370c-7951-4a22-89bc-596ef49a2ad4\") " pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:49.713710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.711766    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/76cab81e-4e12-45f8-b53c-2134434c8663-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-pgb6x\" (UID: \"76cab81e-4e12-45f8-b53c-2134434c8663\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:49.740178 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.740147    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rtxtk\""
Jul 27 17:21:49.747642 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.747618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:49.777700 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.777678    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bvxqm\""
Jul 27 17:21:49.783592 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.783573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-k55f8\""
Jul 27 17:21:49.785688 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.785666    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"
Jul 27 17:21:49.791323 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.791287    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"
Jul 27 17:21:49.834614 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.834588    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4fhbz\""
Jul 27 17:21:49.842067 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.842047    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:49.879599 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.879536    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gv7q2\""
Jul 27 17:21:49.886787 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.886764    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"
Jul 27 17:21:49.907921 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.907896    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2k848\""
Jul 27 17:21:49.915431 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:49.915414    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:21:51.165005 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.164863    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x"]
Jul 27 17:21:51.180685 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:51.180316    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76cab81e_4e12_45f8_b53c_2134434c8663.slice/crio-4a80270ce749c17cb2270b4f071faf92de6536e3f2159dd257baf1bcde3855f1 WatchSource:0}: Error finding container 4a80270ce749c17cb2270b4f071faf92de6536e3f2159dd257baf1bcde3855f1: Status 404 returned error can't find the container with id 4a80270ce749c17cb2270b4f071faf92de6536e3f2159dd257baf1bcde3855f1
Jul 27 17:21:51.259828 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.259765    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86fbf99f7-pzptv"]
Jul 27 17:21:51.264833 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:51.264800    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33096a88_bbfe_4519_b239_157fa4382b21.slice/crio-f8169b408825369c16ddfb16bdfe09e17d4263f530a8a7f172916758798a05c3 WatchSource:0}: Error finding container f8169b408825369c16ddfb16bdfe09e17d4263f530a8a7f172916758798a05c3: Status 404 returned error can't find the container with id f8169b408825369c16ddfb16bdfe09e17d4263f530a8a7f172916758798a05c3
Jul 27 17:21:51.272654 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.272632    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58fd4bb687-9jltm"]
Jul 27 17:21:51.273948 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.273920    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp"]
Jul 27 17:21:51.277583 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.277563    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xxzzn"]
Jul 27 17:21:51.281929 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:51.281903    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d428abe_51d0_42d3_b610_0552dd60f476.slice/crio-bb15f52c196de65b7e84b3b342b26b1bf14f81c1e2d06c5b4235fd00ac550a85 WatchSource:0}: Error finding container bb15f52c196de65b7e84b3b342b26b1bf14f81c1e2d06c5b4235fd00ac550a85: Status 404 returned error can't find the container with id bb15f52c196de65b7e84b3b342b26b1bf14f81c1e2d06c5b4235fd00ac550a85
Jul 27 17:21:51.283026 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:51.283005    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9b370c_7951_4a22_89bc_596ef49a2ad4.slice/crio-9f0599e45effe461570ce052aaf140b7b4a738f16ea7dc38d87a4c89337f5cf5 WatchSource:0}: Error finding container 9f0599e45effe461570ce052aaf140b7b4a738f16ea7dc38d87a4c89337f5cf5: Status 404 returned error can't find the container with id 9f0599e45effe461570ce052aaf140b7b4a738f16ea7dc38d87a4c89337f5cf5
Jul 27 17:21:51.284334 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:51.284122    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92430eb1_2164_4d31_90dc_d80938eaacb7.slice/crio-d9e3244e538bb46e106dcad526f7e9ec59f410381e8816aec77f5f7953ed06aa WatchSource:0}: Error finding container d9e3244e538bb46e106dcad526f7e9ec59f410381e8816aec77f5f7953ed06aa: Status 404 returned error can't find the container with id d9e3244e538bb46e106dcad526f7e9ec59f410381e8816aec77f5f7953ed06aa
Jul 27 17:21:51.474400 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.474319    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l288b"]
Jul 27 17:21:51.477543 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.477515    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bdtq8"]
Jul 27 17:21:51.477847 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:51.477819    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e307bf_7bb5_44dc_8dc0_890bbafffb78.slice/crio-d2547786f7299f1f716f75fcadbf13dd2dd77f50417374fef0fcdea882f8e12b WatchSource:0}: Error finding container d2547786f7299f1f716f75fcadbf13dd2dd77f50417374fef0fcdea882f8e12b: Status 404 returned error can't find the container with id d2547786f7299f1f716f75fcadbf13dd2dd77f50417374fef0fcdea882f8e12b
Jul 27 17:21:51.480937 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:51.480912    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ec967f_de07_46cc_899c_c66c49a81afd.slice/crio-cba1785962976ed062d1812716b7a650eeca5c1705ef5be60292a933891f2baa WatchSource:0}: Error finding container cba1785962976ed062d1812716b7a650eeca5c1705ef5be60292a933891f2baa: Status 404 returned error can't find the container with id cba1785962976ed062d1812716b7a650eeca5c1705ef5be60292a933891f2baa
Jul 27 17:21:51.520811 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.520780    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dls6k"]
Jul 27 17:21:51.522868 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.522843    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt"]
Jul 27 17:21:51.585336 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.585289    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt" event={"ID":"5410afce-d2a3-4357-9959-fd06d74ec212","Type":"ContainerStarted","Data":"5b2d296ea23ae80a81ffc46f6d7af734e762cc8418e2a96a5eebdcb2b0f4da12"}
Jul 27 17:21:51.586853 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.586823    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x" event={"ID":"76cab81e-4e12-45f8-b53c-2134434c8663","Type":"ContainerStarted","Data":"4a80270ce749c17cb2270b4f071faf92de6536e3f2159dd257baf1bcde3855f1"}
Jul 27 17:21:51.588010 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.587981    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dls6k" event={"ID":"12fa5ebb-f199-4903-8d8d-d571608e044b","Type":"ContainerStarted","Data":"3afcb743f05cc099181d6046fadb573fbf2f0f96996da4dee3b3498406d8c63e"}
Jul 27 17:21:51.589416 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.589392    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bdtq8" event={"ID":"c0ec967f-de07-46cc-899c-c66c49a81afd","Type":"ContainerStarted","Data":"8255adecac87ef73feb505379a7315dc19550489176423db152b864bf97b84d0"}
Jul 27 17:21:51.589526 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.589434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bdtq8" event={"ID":"c0ec967f-de07-46cc-899c-c66c49a81afd","Type":"ContainerStarted","Data":"cba1785962976ed062d1812716b7a650eeca5c1705ef5be60292a933891f2baa"}
Jul 27 17:21:51.589526 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.589499    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:21:51.590840 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.590819    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58fd4bb687-9jltm" event={"ID":"9d428abe-51d0-42d3-b610-0552dd60f476","Type":"ContainerStarted","Data":"e64768b5e4fe2d6dda4b7c96b67830fe96c4e3562e6c2647f033043c1338d853"}
Jul 27 17:21:51.590945 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.590854    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58fd4bb687-9jltm" event={"ID":"9d428abe-51d0-42d3-b610-0552dd60f476","Type":"ContainerStarted","Data":"bb15f52c196de65b7e84b3b342b26b1bf14f81c1e2d06c5b4235fd00ac550a85"}
Jul 27 17:21:51.592398 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.592375    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv" event={"ID":"33096a88-bbfe-4519-b239-157fa4382b21","Type":"ContainerStarted","Data":"95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01"}
Jul 27 17:21:51.592487 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.592417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv" event={"ID":"33096a88-bbfe-4519-b239-157fa4382b21","Type":"ContainerStarted","Data":"f8169b408825369c16ddfb16bdfe09e17d4263f530a8a7f172916758798a05c3"}
Jul 27 17:21:51.592487 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.592444    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:21:51.593893 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.593871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zj7b" event={"ID":"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5","Type":"ContainerStarted","Data":"a24e8fd9a4910c516f240ed7d2a600b2c35eec5b783366288642979658cbf7cf"}
Jul 27 17:21:51.595014 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.594992    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xxzzn" event={"ID":"ee9b370c-7951-4a22-89bc-596ef49a2ad4","Type":"ContainerStarted","Data":"9f0599e45effe461570ce052aaf140b7b4a738f16ea7dc38d87a4c89337f5cf5"}
Jul 27 17:21:51.596980 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.596961    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:21:51.597086 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.597061    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" event={"ID":"fb075801-f23d-4b07-a3e8-35b0dc2bda86","Type":"ContainerStarted","Data":"a5224474f4dda2bfcf81e62d608f784dd9e01388a2ed17f9ebed21d67142a871"}
Jul 27 17:21:51.597349 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.597327    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:51.598196 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.598155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l288b" event={"ID":"e4e307bf-7bb5-44dc-8dc0-890bbafffb78","Type":"ContainerStarted","Data":"d2547786f7299f1f716f75fcadbf13dd2dd77f50417374fef0fcdea882f8e12b"}
Jul 27 17:21:51.599349 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.599331    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp" event={"ID":"92430eb1-2164-4d31-90dc-d80938eaacb7","Type":"ContainerStarted","Data":"d9e3244e538bb46e106dcad526f7e9ec59f410381e8816aec77f5f7953ed06aa"}
Jul 27 17:21:51.607449 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.607409    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bdtq8" podStartSLOduration=68.607397165 podStartE2EDuration="1m8.607397165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:51.60652507 +0000 UTC m=+68.867500268" watchObservedRunningTime="2026-07-27 17:21:51.607397165 +0000 UTC m=+68.868372365"
Jul 27 17:21:51.629232 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.629179    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-58fd4bb687-9jltm" podStartSLOduration=58.629159512 podStartE2EDuration="58.629159512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:51.627813099 +0000 UTC m=+68.888788303" watchObservedRunningTime="2026-07-27 17:21:51.629159512 +0000 UTC m=+68.890134714"
Jul 27 17:21:51.647839 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.647639    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv" podStartSLOduration=68.647622546 podStartE2EDuration="1m8.647622546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:51.647467926 +0000 UTC m=+68.908443126" watchObservedRunningTime="2026-07-27 17:21:51.647622546 +0000 UTC m=+68.908597744"
Jul 27 17:21:51.666779 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.666729    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6" podStartSLOduration=48.267649643 podStartE2EDuration="58.666714646s" totalImagesPullingTime="10.399065003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:21:51.666484507 +0000 UTC m=+68.927459706" watchObservedRunningTime="2026-07-27 17:21:51.666714646 +0000 UTC m=+68.927689845"
Jul 27 17:21:51.843057 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.843026    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:51.845892 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:51.845866    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:52.227288 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:52.227218    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-xjqb6"
Jul 27 17:21:52.615134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:52.615034    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:52.622432 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:52.622313    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58fd4bb687-9jltm"
Jul 27 17:21:56.008318 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.008271    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"]
Jul 27 17:21:56.049382 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.049055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"]
Jul 27 17:21:56.049382 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.049198    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:56.053016 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.052963    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 17:21:56.053016 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.052989    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w7cjs\""
Jul 27 17:21:56.054740 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.054541    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 17:21:56.054740 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.054581    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 17:21:56.054740 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.054679    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 27 17:21:56.173012 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.172974    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3a92b080-4bde-47d1-ab28-004044662fce-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-869ff95466-fggpl\" (UID: \"3a92b080-4bde-47d1-ab28-004044662fce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:56.173180 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.173063    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2d8c\" (UniqueName: \"kubernetes.io/projected/3a92b080-4bde-47d1-ab28-004044662fce-kube-api-access-f2d8c\") pod \"managed-serviceaccount-addon-agent-869ff95466-fggpl\" (UID: \"3a92b080-4bde-47d1-ab28-004044662fce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:56.274305 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.274209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2d8c\" (UniqueName: \"kubernetes.io/projected/3a92b080-4bde-47d1-ab28-004044662fce-kube-api-access-f2d8c\") pod \"managed-serviceaccount-addon-agent-869ff95466-fggpl\" (UID: \"3a92b080-4bde-47d1-ab28-004044662fce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:56.274305 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.274262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3a92b080-4bde-47d1-ab28-004044662fce-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-869ff95466-fggpl\" (UID: \"3a92b080-4bde-47d1-ab28-004044662fce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:56.277114 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.277082    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3a92b080-4bde-47d1-ab28-004044662fce-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-869ff95466-fggpl\" (UID: \"3a92b080-4bde-47d1-ab28-004044662fce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:56.299070 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.299038    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2d8c\" (UniqueName: \"kubernetes.io/projected/3a92b080-4bde-47d1-ab28-004044662fce-kube-api-access-f2d8c\") pod \"managed-serviceaccount-addon-agent-869ff95466-fggpl\" (UID: \"3a92b080-4bde-47d1-ab28-004044662fce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:56.368902 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:56.368867    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"
Jul 27 17:21:59.790655 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:21:59.790630    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl"]
Jul 27 17:21:59.959459 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:21:59.959425    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a92b080_4bde_47d1_ab28_004044662fce.slice/crio-06f3bbf3d890addb2abaa3652d9fccc2a5f8971ed0026e19c74dcb659f62f8af WatchSource:0}: Error finding container 06f3bbf3d890addb2abaa3652d9fccc2a5f8971ed0026e19c74dcb659f62f8af: Status 404 returned error can't find the container with id 06f3bbf3d890addb2abaa3652d9fccc2a5f8971ed0026e19c74dcb659f62f8af
Jul 27 17:22:00.645781 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.645750    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l288b" event={"ID":"e4e307bf-7bb5-44dc-8dc0-890bbafffb78","Type":"ContainerStarted","Data":"7157628ee254a2ba783968ce1a37f262cb08e02b5afdce39bd841681d8667a48"}
Jul 27 17:22:00.647337 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.647279    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp" event={"ID":"92430eb1-2164-4d31-90dc-d80938eaacb7","Type":"ContainerStarted","Data":"ee9e2569486f8dc7fda099f9d41e55cf9db815416357fac25938fc8b278b20d1"}
Jul 27 17:22:00.648335 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.648313    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl" event={"ID":"3a92b080-4bde-47d1-ab28-004044662fce","Type":"ContainerStarted","Data":"06f3bbf3d890addb2abaa3652d9fccc2a5f8971ed0026e19c74dcb659f62f8af"}
Jul 27 17:22:00.649971 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.649949    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt" event={"ID":"5410afce-d2a3-4357-9959-fd06d74ec212","Type":"ContainerStarted","Data":"584c1e0ba7fc258140bc075ab61b0bb346c2306b88288649dd8afb09d6117ba3"}
Jul 27 17:22:00.650077 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.649993    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt" event={"ID":"5410afce-d2a3-4357-9959-fd06d74ec212","Type":"ContainerStarted","Data":"963eba9be1d967cfe3694c1df9b4f627f582442140bd4e6861bfd8adbb59522d"}
Jul 27 17:22:00.651415 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.651391    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x" event={"ID":"76cab81e-4e12-45f8-b53c-2134434c8663","Type":"ContainerStarted","Data":"2799f01c860f006836c5fdfe675a8d8993ce9ca19a6a69b1556ce2d05d0dd715"}
Jul 27 17:22:00.652907 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.652885    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dls6k" event={"ID":"12fa5ebb-f199-4903-8d8d-d571608e044b","Type":"ContainerStarted","Data":"7288c6cd8c255a4917b6d3043aedd5708f20834ea1bfd7a447d9282cb26b71b7"}
Jul 27 17:22:00.653028 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.652930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dls6k" event={"ID":"12fa5ebb-f199-4903-8d8d-d571608e044b","Type":"ContainerStarted","Data":"c944d5ce3e7d9f3eab043e4fa26eab19512e8bdd52c54dcb4c74fade35c12f2d"}
Jul 27 17:22:00.654571 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.654552    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zj7b" event={"ID":"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5","Type":"ContainerStarted","Data":"44f37666f5fa1628c1da2b7fb41182ebc19c53a1835ef0806799825d40020bbd"}
Jul 27 17:22:00.656332 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.656315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xxzzn" event={"ID":"ee9b370c-7951-4a22-89bc-596ef49a2ad4","Type":"ContainerStarted","Data":"f84ff1f3a6cac574844b9203c6b1f3454391cc4be8ec93d65f56e63ceea37478"}
Jul 27 17:22:00.656417 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.656347    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xxzzn" event={"ID":"ee9b370c-7951-4a22-89bc-596ef49a2ad4","Type":"ContainerStarted","Data":"7163e90784a4eddb4428d7e78f32ea90870ffc6ff7c6ce47fb72419576b03e3f"}
Jul 27 17:22:00.656417 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.656368    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:22:00.661180 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.661145    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l288b" podStartSLOduration=69.488256956 podStartE2EDuration="1m17.661134944s" totalImagesPullingTime="8.172877988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:00.660986165 +0000 UTC m=+77.921961363" watchObservedRunningTime="2026-07-27 17:22:00.661134944 +0000 UTC m=+77.922110143"
Jul 27 17:22:00.680687 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.680649    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-947dp" podStartSLOduration=59.353986099 podStartE2EDuration="1m7.680637453s" totalImagesPullingTime="8.326651354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:00.679998064 +0000 UTC m=+77.940973264" watchObservedRunningTime="2026-07-27 17:22:00.680637453 +0000 UTC m=+77.941612652"
Jul 27 17:22:00.697906 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.697867    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-l8qgt" podStartSLOduration=59.630386336 podStartE2EDuration="1m7.69785315s" totalImagesPullingTime="8.067466814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:00.696427323 +0000 UTC m=+77.957402528" watchObservedRunningTime="2026-07-27 17:22:00.69785315 +0000 UTC m=+77.958828351"
Jul 27 17:22:00.717068 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.717022    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-pgb6x" podStartSLOduration=61.650960126 podStartE2EDuration="1m7.717005208s" totalImagesPullingTime="6.066045082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:00.716549874 +0000 UTC m=+77.977525074" watchObservedRunningTime="2026-07-27 17:22:00.717005208 +0000 UTC m=+77.977980408"
Jul 27 17:22:00.736719 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.736668    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xxzzn" podStartSLOduration=35.539355715 podStartE2EDuration="43.736651337s" totalImagesPullingTime="8.197295622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:00.735512024 +0000 UTC m=+77.996487224" watchObservedRunningTime="2026-07-27 17:22:00.736651337 +0000 UTC m=+77.997626535"
Jul 27 17:22:00.752833 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:00.752791    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dls6k" podStartSLOduration=69.673020029 podStartE2EDuration="1m17.752780558s" totalImagesPullingTime="8.079760529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:00.7515078 +0000 UTC m=+78.012483001" watchObservedRunningTime="2026-07-27 17:22:00.752780558 +0000 UTC m=+78.013755757"
Jul 27 17:22:05.677483 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:05.677442    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl" event={"ID":"3a92b080-4bde-47d1-ab28-004044662fce","Type":"ContainerStarted","Data":"d49a7eba6112a93cc5a1bac8fd69ad69966601951fa3e3e7bce855da3acf2499"}
Jul 27 17:22:05.679338 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:05.679311    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zj7b" event={"ID":"830ccf32-fb7c-461b-bf99-4aa55b3ad9a5","Type":"ContainerStarted","Data":"a956c067da62a8a44b577c6327576b818a97474ace9ef2b17a0c3fcea681db46"}
Jul 27 17:22:05.695532 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:05.695491    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869ff95466-fggpl" podStartSLOduration=5.877060964 podStartE2EDuration="10.695478851s" totalImagesPullingTime="4.818417887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:05.695270173 +0000 UTC m=+82.956245372" watchObservedRunningTime="2026-07-27 17:22:05.695478851 +0000 UTC m=+82.956454028"
Jul 27 17:22:05.720904 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:05.720863    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8zj7b" podStartSLOduration=18.269239832 podStartE2EDuration="36.720849067s" totalImagesPullingTime="18.451609235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:21:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:05.71931297 +0000 UTC m=+82.980288160" watchObservedRunningTime="2026-07-27 17:22:05.720849067 +0000 UTC m=+82.981824267"
Jul 27 17:22:09.752815 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:09.752771    2565 patch_prober.go:28] interesting pod/image-registry-86fbf99f7-pzptv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 17:22:09.753212 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:09.752825    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv" podUID="33096a88-bbfe-4519-b239-157fa4382b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 17:22:10.663806 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:10.663774    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xxzzn"
Jul 27 17:22:12.617961 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:12.617935    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:22:18.736345 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.736309    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xrd2m"]
Jul 27 17:22:18.739817 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.739800    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.742256 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.742237    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 17:22:18.742478 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.742462    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k48sk\""
Jul 27 17:22:18.742478 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.742470    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 17:22:18.743545 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.743522    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 17:22:18.743643 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.743570    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 17:22:18.843891 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.843856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-textfile\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.843891 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.843894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-tls\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.844134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.843923    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpx2m\" (UniqueName: \"kubernetes.io/projected/b0c852db-9a78-43a4-8c96-18cac555b90e-kube-api-access-dpx2m\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.844134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.844013    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-wtmp\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.844134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.844058    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.844134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.844093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-sys\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.844373 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.844135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-accelerators-collector-config\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.844373 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.844179    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0c852db-9a78-43a4-8c96-18cac555b90e-metrics-client-ca\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.844373 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.844231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-root\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945227 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945192    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-wtmp\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945227 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945255    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-sys\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-accelerators-collector-config\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945349    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0c852db-9a78-43a4-8c96-18cac555b90e-metrics-client-ca\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945395    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-wtmp\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945398    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-root\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945437    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-root\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945396    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0c852db-9a78-43a4-8c96-18cac555b90e-sys\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945764 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-textfile\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945764 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-tls\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945764 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945558    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpx2m\" (UniqueName: \"kubernetes.io/projected/b0c852db-9a78-43a4-8c96-18cac555b90e-kube-api-access-dpx2m\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945764 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:22:18.945593    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 17:22:18.945764 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:22:18.945658    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-tls podName:b0c852db-9a78-43a4-8c96-18cac555b90e nodeName:}" failed. No retries permitted until 2026-07-27 17:22:19.445637866 +0000 UTC m=+96.706613043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-tls") pod "node-exporter-xrd2m" (UID: "b0c852db-9a78-43a4-8c96-18cac555b90e") : secret "node-exporter-tls" not found
Jul 27 17:22:18.945949 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-textfile\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.945949 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.945937    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-accelerators-collector-config\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.946023 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.946008    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0c852db-9a78-43a4-8c96-18cac555b90e-metrics-client-ca\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.947616 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.947599    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:18.956149 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:18.956128    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpx2m\" (UniqueName: \"kubernetes.io/projected/b0c852db-9a78-43a4-8c96-18cac555b90e-kube-api-access-dpx2m\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:19.316198 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:19.316165    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86fbf99f7-pzptv"]
Jul 27 17:22:19.449884 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:19.449848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-tls\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:19.452111 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:19.452088    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b0c852db-9a78-43a4-8c96-18cac555b90e-node-exporter-tls\") pod \"node-exporter-xrd2m\" (UID: \"b0c852db-9a78-43a4-8c96-18cac555b90e\") " pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:19.649541 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:19.649460    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xrd2m"
Jul 27 17:22:19.660635 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:22:19.660612    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c852db_9a78_43a4_8c96_18cac555b90e.slice/crio-c0eab5738a2911e9291b00ac5812275d8acd4a10b1b77abda1c31f89da62a372 WatchSource:0}: Error finding container c0eab5738a2911e9291b00ac5812275d8acd4a10b1b77abda1c31f89da62a372: Status 404 returned error can't find the container with id c0eab5738a2911e9291b00ac5812275d8acd4a10b1b77abda1c31f89da62a372
Jul 27 17:22:19.734590 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:19.734555    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrd2m" event={"ID":"b0c852db-9a78-43a4-8c96-18cac555b90e","Type":"ContainerStarted","Data":"c0eab5738a2911e9291b00ac5812275d8acd4a10b1b77abda1c31f89da62a372"}
Jul 27 17:22:21.745871 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:21.745829    2565 generic.go:356] "Generic (PLEG): container finished" podID="b0c852db-9a78-43a4-8c96-18cac555b90e" containerID="6ba6b034ba884fcfda80db73e60e76a583b017f8a10fd69af740d0ab686a2d20" exitCode=0
Jul 27 17:22:21.746344 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:21.745876    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrd2m" event={"ID":"b0c852db-9a78-43a4-8c96-18cac555b90e","Type":"ContainerDied","Data":"6ba6b034ba884fcfda80db73e60e76a583b017f8a10fd69af740d0ab686a2d20"}
Jul 27 17:22:22.616525 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:22.616493    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bdtq8"
Jul 27 17:22:22.752254 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:22.752212    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrd2m" event={"ID":"b0c852db-9a78-43a4-8c96-18cac555b90e","Type":"ContainerStarted","Data":"f34ab34b33f1d949aac757906e0de2d19ceaf2b16e7d31197bf115e136eb6fc0"}
Jul 27 17:22:22.752621 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:22.752278    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xrd2m" event={"ID":"b0c852db-9a78-43a4-8c96-18cac555b90e","Type":"ContainerStarted","Data":"d6e9e93cd1a8b418090ce9e022a75cc8d0adde9dfd251e2569b3c64375314c27"}
Jul 27 17:22:22.773199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:22.773148    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xrd2m" podStartSLOduration=3.171817806 podStartE2EDuration="4.77313182s" totalImagesPullingTime="1.515370568s" totalInitContainerRuntime="85.943446ms" isStatefulPod=true podCreationTimestamp="2026-07-27 17:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:22:22.771898615 +0000 UTC m=+100.032873815" watchObservedRunningTime="2026-07-27 17:22:22.77313182 +0000 UTC m=+100.034107018"
Jul 27 17:22:35.800712 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:35.800677    2565 generic.go:356] "Generic (PLEG): container finished" podID="1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be" containerID="eeeec3086b8e34a9028561df384c48562179ea0cb1396bbb2f28eeee8f57266c" exitCode=0
Jul 27 17:22:35.801127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:35.800705    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9" event={"ID":"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be","Type":"ContainerDied","Data":"eeeec3086b8e34a9028561df384c48562179ea0cb1396bbb2f28eeee8f57266c"}
Jul 27 17:22:35.801127 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:35.801024    2565 scope.go:117] "RemoveContainer" containerID="eeeec3086b8e34a9028561df384c48562179ea0cb1396bbb2f28eeee8f57266c"
Jul 27 17:22:36.806961 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:36.806919    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9" event={"ID":"1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be","Type":"ContainerStarted","Data":"a4efca82ead88cf59b8b9be406636fce096242cd53c277e8749309b881950b69"}
Jul 27 17:22:39.817849 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:39.817820    2565 generic.go:356] "Generic (PLEG): container finished" podID="668f942d-f6dc-4152-9bd0-870aa27229ba" containerID="d23c80deb5ca58b697019d6689240862403133b09cf636cc827ea306049145bf" exitCode=0
Jul 27 17:22:39.818229 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:39.817889    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj" event={"ID":"668f942d-f6dc-4152-9bd0-870aa27229ba","Type":"ContainerDied","Data":"d23c80deb5ca58b697019d6689240862403133b09cf636cc827ea306049145bf"}
Jul 27 17:22:39.818229 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:39.818170    2565 scope.go:117] "RemoveContainer" containerID="d23c80deb5ca58b697019d6689240862403133b09cf636cc827ea306049145bf"
Jul 27 17:22:40.823236 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:40.823198    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bhxjj" event={"ID":"668f942d-f6dc-4152-9bd0-870aa27229ba","Type":"ContainerStarted","Data":"71a6b160762b15ad1e5e414ccf382a68010e640920ca896ec70100e9a83fe2a4"}
Jul 27 17:22:44.334910 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.334862    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv" podUID="33096a88-bbfe-4519-b239-157fa4382b21" containerName="registry" containerID="cri-o://95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01" gracePeriod=30
Jul 27 17:22:44.574070 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.574047    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:22:44.643264 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643199    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-bound-sa-token\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643264 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643239    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdsb7\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-kube-api-access-kdsb7\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643278    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-registry-certificates\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643328    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643384    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-image-registry-private-configuration\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643421    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33096a88-bbfe-4519-b239-157fa4382b21-ca-trust-extracted\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643443    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-trusted-ca\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643478    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-installation-pull-secrets\") pod \"33096a88-bbfe-4519-b239-157fa4382b21\" (UID: \"33096a88-bbfe-4519-b239-157fa4382b21\") "
Jul 27 17:22:44.643831 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.643805    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:44.644268 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.644241    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:22:44.645902 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.645855    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:22:44.646067 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.646045    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-kube-api-access-kdsb7" (OuterVolumeSpecName: "kube-api-access-kdsb7") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "kube-api-access-kdsb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:22:44.646067 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.646054    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:22:44.646197 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.646112    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:22:44.646197 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.646136    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:22:44.652059 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.652034    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33096a88-bbfe-4519-b239-157fa4382b21-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "33096a88-bbfe-4519-b239-157fa4382b21" (UID: "33096a88-bbfe-4519-b239-157fa4382b21"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:22:44.744604 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744576    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdsb7\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-kube-api-access-kdsb7\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.744604 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744597    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-registry-certificates\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.744731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744607    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-registry-tls\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.744731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744618    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-image-registry-private-configuration\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.744731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744627    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33096a88-bbfe-4519-b239-157fa4382b21-ca-trust-extracted\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.744731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744635    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33096a88-bbfe-4519-b239-157fa4382b21-trusted-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.744731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744643    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33096a88-bbfe-4519-b239-157fa4382b21-installation-pull-secrets\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.744731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.744651    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33096a88-bbfe-4519-b239-157fa4382b21-bound-sa-token\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:22:44.842802 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.842767    2565 generic.go:356] "Generic (PLEG): container finished" podID="33096a88-bbfe-4519-b239-157fa4382b21" containerID="95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01" exitCode=0
Jul 27 17:22:44.842981 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.842833    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv"
Jul 27 17:22:44.842981 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.842829    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv" event={"ID":"33096a88-bbfe-4519-b239-157fa4382b21","Type":"ContainerDied","Data":"95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01"}
Jul 27 17:22:44.843090 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.842982    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86fbf99f7-pzptv" event={"ID":"33096a88-bbfe-4519-b239-157fa4382b21","Type":"ContainerDied","Data":"f8169b408825369c16ddfb16bdfe09e17d4263f530a8a7f172916758798a05c3"}
Jul 27 17:22:44.843090 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.843005    2565 scope.go:117] "RemoveContainer" containerID="95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01"
Jul 27 17:22:44.851755 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.851736    2565 scope.go:117] "RemoveContainer" containerID="95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01"
Jul 27 17:22:44.851999 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:22:44.851978    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01\": container with ID starting with 95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01 not found: ID does not exist" containerID="95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01"
Jul 27 17:22:44.852067 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.852006    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01"} err="failed to get container status \"95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01\": rpc error: code = NotFound desc = could not find container \"95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01\": container with ID starting with 95143c1f0df965a62822d4046df5773ebcb680c82257805812c10ee7bc421c01 not found: ID does not exist"
Jul 27 17:22:44.864224 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.864202    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86fbf99f7-pzptv"]
Jul 27 17:22:44.867878 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:44.867858    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86fbf99f7-pzptv"]
Jul 27 17:22:45.254718 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:45.254686    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33096a88-bbfe-4519-b239-157fa4382b21" path="/var/lib/kubelet/pods/33096a88-bbfe-4519-b239-157fa4382b21/volumes"
Jul 27 17:22:54.882256 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:54.882215    2565 generic.go:356] "Generic (PLEG): container finished" podID="76f6fe48-c826-4af3-bc3d-ab70010dcb93" containerID="a3b7f68aee04a7d2f5a053b1ab717454cd74c3bdc30a02d34a7230c79227fb0e" exitCode=0
Jul 27 17:22:54.882653 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:54.882306    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mqz4z" event={"ID":"76f6fe48-c826-4af3-bc3d-ab70010dcb93","Type":"ContainerDied","Data":"a3b7f68aee04a7d2f5a053b1ab717454cd74c3bdc30a02d34a7230c79227fb0e"}
Jul 27 17:22:54.882740 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:54.882666    2565 scope.go:117] "RemoveContainer" containerID="a3b7f68aee04a7d2f5a053b1ab717454cd74c3bdc30a02d34a7230c79227fb0e"
Jul 27 17:22:55.887976 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:22:55.887943    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mqz4z" event={"ID":"76f6fe48-c826-4af3-bc3d-ab70010dcb93","Type":"ContainerStarted","Data":"18cf38d65183379d492f053d8db64e626fabfc8806acd6c6fc6912f5c7b874a1"}
Jul 27 17:23:43.190158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.190128    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"]
Jul 27 17:23:43.190542 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.190428    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33096a88-bbfe-4519-b239-157fa4382b21" containerName="registry"
Jul 27 17:23:43.190542 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.190443    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="33096a88-bbfe-4519-b239-157fa4382b21" containerName="registry"
Jul 27 17:23:43.190542 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.190521    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="33096a88-bbfe-4519-b239-157fa4382b21" containerName="registry"
Jul 27 17:23:43.193475 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.193457    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.196187 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.196171    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 27 17:23:43.196277 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.196212    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c6k2r\""
Jul 27 17:23:43.196602 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.196588    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 27 17:23:43.196654 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.196639    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 27 17:23:43.196818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.196804    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 27 17:23:43.197044 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.197028    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 27 17:23:43.203080 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.203061    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 27 17:23:43.209138 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.209115    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"]
Jul 27 17:23:43.268503 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268480    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfkgq\" (UniqueName: \"kubernetes.io/projected/3da258f4-720e-4bc8-992f-2e73584c4948-kube-api-access-qfkgq\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.268627 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268510    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-secret-telemeter-client\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.268627 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268547    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-telemeter-trusted-ca-bundle\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.268739 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-telemeter-client-tls\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.268794 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268741    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.268848 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268799    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-serving-certs-ca-bundle\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.268848 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268831    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-federate-client-tls\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.268938 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.268860    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-metrics-client-ca\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.369963 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.369873    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-federate-client-tls\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.369963 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.369923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-metrics-client-ca\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.369963 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.369944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfkgq\" (UniqueName: \"kubernetes.io/projected/3da258f4-720e-4bc8-992f-2e73584c4948-kube-api-access-qfkgq\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.369963 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.369968    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-secret-telemeter-client\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.370264 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.369996    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-telemeter-trusted-ca-bundle\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.370264 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.370023    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-telemeter-client-tls\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.370264 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.370048    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.370264 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.370082    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-serving-certs-ca-bundle\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.370731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.370706    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-metrics-client-ca\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.370857 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.370802    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-serving-certs-ca-bundle\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.371397 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.371371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3da258f4-720e-4bc8-992f-2e73584c4948-telemeter-trusted-ca-bundle\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.372465 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.372441    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-federate-client-tls\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.372636 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.372617    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-telemeter-client-tls\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.372694 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.372677    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-secret-telemeter-client\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.372738 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.372722    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3da258f4-720e-4bc8-992f-2e73584c4948-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.378284 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.378267    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfkgq\" (UniqueName: \"kubernetes.io/projected/3da258f4-720e-4bc8-992f-2e73584c4948-kube-api-access-qfkgq\") pod \"telemeter-client-657c6c8b9d-5lgw7\" (UID: \"3da258f4-720e-4bc8-992f-2e73584c4948\") " pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.502377 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.502349    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"
Jul 27 17:23:43.637151 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:43.637125    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7"]
Jul 27 17:23:43.638923 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:23:43.638895    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da258f4_720e_4bc8_992f_2e73584c4948.slice/crio-e686e23a62b0ce79acfe18fd3fb45e7ff32ee7fd3cdb60a8eeec06b4d04d5842 WatchSource:0}: Error finding container e686e23a62b0ce79acfe18fd3fb45e7ff32ee7fd3cdb60a8eeec06b4d04d5842: Status 404 returned error can't find the container with id e686e23a62b0ce79acfe18fd3fb45e7ff32ee7fd3cdb60a8eeec06b4d04d5842
Jul 27 17:23:44.066913 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:44.066883    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7" event={"ID":"3da258f4-720e-4bc8-992f-2e73584c4948","Type":"ContainerStarted","Data":"e686e23a62b0ce79acfe18fd3fb45e7ff32ee7fd3cdb60a8eeec06b4d04d5842"}
Jul 27 17:23:46.076733 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:46.076698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7" event={"ID":"3da258f4-720e-4bc8-992f-2e73584c4948","Type":"ContainerStarted","Data":"6caeb8d6b65d71b0acf210c3935faf92b66a79fdb7f9dc2c366ce383aafd0e1d"}
Jul 27 17:23:47.082626 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:47.082591    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7" event={"ID":"3da258f4-720e-4bc8-992f-2e73584c4948","Type":"ContainerStarted","Data":"d1b180b22c784804299b7c415cca3a444812cd34eefad1239c6fe45a640c9965"}
Jul 27 17:23:47.082997 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:47.082651    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7" event={"ID":"3da258f4-720e-4bc8-992f-2e73584c4948","Type":"ContainerStarted","Data":"4cc629d2fd57019ae8e3eefa100026ebd1783566488e77447c5a2efb6eb2a744"}
Jul 27 17:23:47.110723 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:23:47.110676    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-657c6c8b9d-5lgw7" podStartSLOduration=1.173355168 podStartE2EDuration="4.11066145s" totalImagesPullingTime="2.937306282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:23:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:23:47.108876955 +0000 UTC m=+184.369852192" watchObservedRunningTime="2026-07-27 17:23:47.11066145 +0000 UTC m=+184.371636648"
Jul 27 17:25:43.117574 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:43.117543    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:25:43.123254 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:43.123232    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:25:43.127687 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:43.127667    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 17:25:52.564424 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.564382    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"]
Jul 27 17:25:52.567404 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.567388    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:52.570260 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.570240    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 17:25:52.571624 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.571595    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 17:25:52.571624 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.571598    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xqfrl\""
Jul 27 17:25:52.577230 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.577210    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"]
Jul 27 17:25:52.676841 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.676813    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4719f8e-36ca-400a-bb22-71061d67feb4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ghjlf\" (UID: \"a4719f8e-36ca-400a-bb22-71061d67feb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:52.677058 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.676856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7swg2\" (UniqueName: \"kubernetes.io/projected/a4719f8e-36ca-400a-bb22-71061d67feb4-kube-api-access-7swg2\") pod \"cert-manager-cainjector-559cdd8c87-ghjlf\" (UID: \"a4719f8e-36ca-400a-bb22-71061d67feb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:52.777844 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.777819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4719f8e-36ca-400a-bb22-71061d67feb4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ghjlf\" (UID: \"a4719f8e-36ca-400a-bb22-71061d67feb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:52.777954 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.777862    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7swg2\" (UniqueName: \"kubernetes.io/projected/a4719f8e-36ca-400a-bb22-71061d67feb4-kube-api-access-7swg2\") pod \"cert-manager-cainjector-559cdd8c87-ghjlf\" (UID: \"a4719f8e-36ca-400a-bb22-71061d67feb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:52.786595 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.786572    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4719f8e-36ca-400a-bb22-71061d67feb4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ghjlf\" (UID: \"a4719f8e-36ca-400a-bb22-71061d67feb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:52.787620 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.787601    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7swg2\" (UniqueName: \"kubernetes.io/projected/a4719f8e-36ca-400a-bb22-71061d67feb4-kube-api-access-7swg2\") pod \"cert-manager-cainjector-559cdd8c87-ghjlf\" (UID: \"a4719f8e-36ca-400a-bb22-71061d67feb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:52.876690 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:52.876633    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"
Jul 27 17:25:53.004186 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:53.004162    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf"]
Jul 27 17:25:53.006566 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:25:53.006539    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4719f8e_36ca_400a_bb22_71061d67feb4.slice/crio-f366fee9b7ec52b9899b61ad2a3a865cb46bc048e6848ea233523739cab0eb15 WatchSource:0}: Error finding container f366fee9b7ec52b9899b61ad2a3a865cb46bc048e6848ea233523739cab0eb15: Status 404 returned error can't find the container with id f366fee9b7ec52b9899b61ad2a3a865cb46bc048e6848ea233523739cab0eb15
Jul 27 17:25:53.008425 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:53.008407    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:25:53.535769 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:25:53.535738    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf" event={"ID":"a4719f8e-36ca-400a-bb22-71061d67feb4","Type":"ContainerStarted","Data":"f366fee9b7ec52b9899b61ad2a3a865cb46bc048e6848ea233523739cab0eb15"}
Jul 27 17:26:02.570702 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:02.570666    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf" event={"ID":"a4719f8e-36ca-400a-bb22-71061d67feb4","Type":"ContainerStarted","Data":"97179e1f8697f1bc2d2c22ec30cbc8d4b00407773fd361ef3d71e2a19fe77b95"}
Jul 27 17:26:02.588592 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:02.588543    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ghjlf" podStartSLOduration=1.6915092330000001 podStartE2EDuration="10.58852842s" totalImagesPullingTime="8.897019187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:02.588256144 +0000 UTC m=+319.849231345" watchObservedRunningTime="2026-07-27 17:26:02.58852842 +0000 UTC m=+319.849503597"
Jul 27 17:26:24.802321 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.802274    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"]
Jul 27 17:26:24.808480 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.808462    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:24.812836 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.812812    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 17:26:24.812972 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.812868    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 17:26:24.812972 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.812923    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 17:26:24.813080 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.813047    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nqddh\""
Jul 27 17:26:24.813173 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.813154    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 17:26:24.829427 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.829402    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"]
Jul 27 17:26:24.914600 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.914569    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/31466f32-8da3-4df6-a383-d777742f74fc-manifests\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:24.914770 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.914622    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31466f32-8da3-4df6-a383-d777742f74fc-tmp\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:24.914770 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.914677    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp2mt\" (UniqueName: \"kubernetes.io/projected/31466f32-8da3-4df6-a383-d777742f74fc-kube-api-access-wp2mt\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:24.914770 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.914697    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31466f32-8da3-4df6-a383-d777742f74fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:24.914875 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:24.914775    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31466f32-8da3-4df6-a383-d777742f74fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.015186 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.015150    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/31466f32-8da3-4df6-a383-d777742f74fc-manifests\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.015385 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.015203    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31466f32-8da3-4df6-a383-d777742f74fc-tmp\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.015385 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.015231    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp2mt\" (UniqueName: \"kubernetes.io/projected/31466f32-8da3-4df6-a383-d777742f74fc-kube-api-access-wp2mt\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.015385 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.015246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31466f32-8da3-4df6-a383-d777742f74fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.015385 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.015273    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31466f32-8da3-4df6-a383-d777742f74fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.015627 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.015564    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/31466f32-8da3-4df6-a383-d777742f74fc-manifests\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.017588 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.017565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31466f32-8da3-4df6-a383-d777742f74fc-tmp\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.017724 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.017708    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31466f32-8da3-4df6-a383-d777742f74fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.017836 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.017819    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31466f32-8da3-4df6-a383-d777742f74fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.028937 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.028904    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp2mt\" (UniqueName: \"kubernetes.io/projected/31466f32-8da3-4df6-a383-d777742f74fc-kube-api-access-wp2mt\") pod \"opendatahub-operator-controller-manager-746b8d4d47-g8zdl\" (UID: \"31466f32-8da3-4df6-a383-d777742f74fc\") " pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.118996 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.118903    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:25.255955 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.255933    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"]
Jul 27 17:26:25.258450 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:26:25.258424    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31466f32_8da3_4df6_a383_d777742f74fc.slice/crio-0959e090df0e2f3a8d4c77bf29171712d8cc434615fb9726f1a927340b97936b WatchSource:0}: Error finding container 0959e090df0e2f3a8d4c77bf29171712d8cc434615fb9726f1a927340b97936b: Status 404 returned error can't find the container with id 0959e090df0e2f3a8d4c77bf29171712d8cc434615fb9726f1a927340b97936b
Jul 27 17:26:25.661814 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:25.661781    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl" event={"ID":"31466f32-8da3-4df6-a383-d777742f74fc","Type":"ContainerStarted","Data":"0959e090df0e2f3a8d4c77bf29171712d8cc434615fb9726f1a927340b97936b"}
Jul 27 17:26:28.675708 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:28.675673    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl" event={"ID":"31466f32-8da3-4df6-a383-d777742f74fc","Type":"ContainerStarted","Data":"deecdd521c72d171ec9a72c28406d82e168c79dda93f1766807e621693f680fe"}
Jul 27 17:26:29.681357 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:29.681324    2565 generic.go:356] "Generic (PLEG): container finished" podID="31466f32-8da3-4df6-a383-d777742f74fc" containerID="deecdd521c72d171ec9a72c28406d82e168c79dda93f1766807e621693f680fe" exitCode=0
Jul 27 17:26:29.681711 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:29.681403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl" event={"ID":"31466f32-8da3-4df6-a383-d777742f74fc","Type":"ContainerDied","Data":"deecdd521c72d171ec9a72c28406d82e168c79dda93f1766807e621693f680fe"}
Jul 27 17:26:30.686822 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:30.686787    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl" event={"ID":"31466f32-8da3-4df6-a383-d777742f74fc","Type":"ContainerStarted","Data":"bb32fd0b764181311a2a30634f13694e190386b0cd144635a20161633c81464f"}
Jul 27 17:26:30.687234 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:30.686873    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:30.709455 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:30.709407    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl" podStartSLOduration=1.976557165 podStartE2EDuration="6.709391828s" totalImagesPullingTime="3.683355895s" totalInitContainerRuntime="1.049478768s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:30.707134614 +0000 UTC m=+347.968109814" watchObservedRunningTime="2026-07-27 17:26:30.709391828 +0000 UTC m=+347.970367027"
Jul 27 17:26:41.693523 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:41.693437    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-746b8d4d47-g8zdl"
Jul 27 17:26:44.541097 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.541061    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"]
Jul 27 17:26:44.545644 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.545617    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.548993 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.548971    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 17:26:44.550144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.550120    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 17:26:44.567875 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.567855    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"]
Jul 27 17:26:44.577561 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.577541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00da9703-3511-4e59-8048-72e5cd010f14-tls-certs\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.577659 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.577568    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00da9703-3511-4e59-8048-72e5cd010f14-tmp\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.577718 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.577668    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrzxg\" (UniqueName: \"kubernetes.io/projected/00da9703-3511-4e59-8048-72e5cd010f14-kube-api-access-hrzxg\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.678932 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.678900    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00da9703-3511-4e59-8048-72e5cd010f14-tls-certs\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.678932 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.678933    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00da9703-3511-4e59-8048-72e5cd010f14-tmp\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.679187 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.678993    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrzxg\" (UniqueName: \"kubernetes.io/projected/00da9703-3511-4e59-8048-72e5cd010f14-kube-api-access-hrzxg\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.681431 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.681386    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00da9703-3511-4e59-8048-72e5cd010f14-tmp\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.681576 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.681553    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00da9703-3511-4e59-8048-72e5cd010f14-tls-certs\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.695663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.695635    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrzxg\" (UniqueName: \"kubernetes.io/projected/00da9703-3511-4e59-8048-72e5cd010f14-kube-api-access-hrzxg\") pod \"kube-auth-proxy-5b8c977c58-q975s\" (UID: \"00da9703-3511-4e59-8048-72e5cd010f14\") " pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.728467 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.728436    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pjkkx"]
Jul 27 17:26:44.731764 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.731746    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:44.736066 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.736048    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qr2nv\""
Jul 27 17:26:44.736409 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.736396    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 27 17:26:44.756549 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.756517    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pjkkx"]
Jul 27 17:26:44.779921 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.779894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2prn8\" (UniqueName: \"kubernetes.io/projected/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-kube-api-access-2prn8\") pod \"odh-model-controller-7998c67dc4-pjkkx\" (UID: \"821bb5e4-886a-4df1-8c7b-1399e7da2bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:44.780045 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.779954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-cert\") pod \"odh-model-controller-7998c67dc4-pjkkx\" (UID: \"821bb5e4-886a-4df1-8c7b-1399e7da2bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:44.855185 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.855153    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"
Jul 27 17:26:44.880537 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.880512    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2prn8\" (UniqueName: \"kubernetes.io/projected/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-kube-api-access-2prn8\") pod \"odh-model-controller-7998c67dc4-pjkkx\" (UID: \"821bb5e4-886a-4df1-8c7b-1399e7da2bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:44.880650 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.880567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-cert\") pod \"odh-model-controller-7998c67dc4-pjkkx\" (UID: \"821bb5e4-886a-4df1-8c7b-1399e7da2bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:44.880725 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:26:44.880680    2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 27 17:26:44.880772 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:26:44.880742    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-cert podName:821bb5e4-886a-4df1-8c7b-1399e7da2bb3 nodeName:}" failed. No retries permitted until 2026-07-27 17:26:45.380724137 +0000 UTC m=+362.641699314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-cert") pod "odh-model-controller-7998c67dc4-pjkkx" (UID: "821bb5e4-886a-4df1-8c7b-1399e7da2bb3") : secret "odh-model-controller-webhook-cert" not found
Jul 27 17:26:44.894088 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:44.894060    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2prn8\" (UniqueName: \"kubernetes.io/projected/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-kube-api-access-2prn8\") pod \"odh-model-controller-7998c67dc4-pjkkx\" (UID: \"821bb5e4-886a-4df1-8c7b-1399e7da2bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:45.003494 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.003468    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b8c977c58-q975s"]
Jul 27 17:26:45.005362 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:26:45.005337    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00da9703_3511_4e59_8048_72e5cd010f14.slice/crio-e0896b906101fa3f52c4e608e64228d75a223187706f5032ac87d90abe2f7e93 WatchSource:0}: Error finding container e0896b906101fa3f52c4e608e64228d75a223187706f5032ac87d90abe2f7e93: Status 404 returned error can't find the container with id e0896b906101fa3f52c4e608e64228d75a223187706f5032ac87d90abe2f7e93
Jul 27 17:26:45.385585 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.385547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-cert\") pod \"odh-model-controller-7998c67dc4-pjkkx\" (UID: \"821bb5e4-886a-4df1-8c7b-1399e7da2bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:45.387895 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.387867    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/821bb5e4-886a-4df1-8c7b-1399e7da2bb3-cert\") pod \"odh-model-controller-7998c67dc4-pjkkx\" (UID: \"821bb5e4-886a-4df1-8c7b-1399e7da2bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:45.540412 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.540372    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9kg5s"]
Jul 27 17:26:45.545457 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.545422    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:45.549367 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.549343    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 27 17:26:45.560230 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.560205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9kg5s"]
Jul 27 17:26:45.586734 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.586706    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7a4ca3-159a-4f46-9d85-a512339877e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-9kg5s\" (UID: \"ed7a4ca3-159a-4f46-9d85-a512339877e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:45.586866 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.586749    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wbrb\" (UniqueName: \"kubernetes.io/projected/ed7a4ca3-159a-4f46-9d85-a512339877e5-kube-api-access-7wbrb\") pod \"model-serving-api-54d89bd5fc-9kg5s\" (UID: \"ed7a4ca3-159a-4f46-9d85-a512339877e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:45.642159 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.642057    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:45.687446 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.687405    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7a4ca3-159a-4f46-9d85-a512339877e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-9kg5s\" (UID: \"ed7a4ca3-159a-4f46-9d85-a512339877e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:45.687613 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.687473    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrb\" (UniqueName: \"kubernetes.io/projected/ed7a4ca3-159a-4f46-9d85-a512339877e5-kube-api-access-7wbrb\") pod \"model-serving-api-54d89bd5fc-9kg5s\" (UID: \"ed7a4ca3-159a-4f46-9d85-a512339877e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:45.687613 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:26:45.687553    2565 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 27 17:26:45.687728 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:26:45.687630    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7a4ca3-159a-4f46-9d85-a512339877e5-tls-certs podName:ed7a4ca3-159a-4f46-9d85-a512339877e5 nodeName:}" failed. No retries permitted until 2026-07-27 17:26:46.187607999 +0000 UTC m=+363.448583192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ed7a4ca3-159a-4f46-9d85-a512339877e5-tls-certs") pod "model-serving-api-54d89bd5fc-9kg5s" (UID: "ed7a4ca3-159a-4f46-9d85-a512339877e5") : secret "model-serving-api-tls" not found
Jul 27 17:26:45.697395 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.697368    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wbrb\" (UniqueName: \"kubernetes.io/projected/ed7a4ca3-159a-4f46-9d85-a512339877e5-kube-api-access-7wbrb\") pod \"model-serving-api-54d89bd5fc-9kg5s\" (UID: \"ed7a4ca3-159a-4f46-9d85-a512339877e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:45.750712 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.750674    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s" event={"ID":"00da9703-3511-4e59-8048-72e5cd010f14","Type":"ContainerStarted","Data":"e0896b906101fa3f52c4e608e64228d75a223187706f5032ac87d90abe2f7e93"}
Jul 27 17:26:45.774316 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:45.774269    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pjkkx"]
Jul 27 17:26:45.776333 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:26:45.776305    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821bb5e4_886a_4df1_8c7b_1399e7da2bb3.slice/crio-79e3c2c189fddd135fcaee4822e154305ade87fbe16bc1a58818548ec8394b9a WatchSource:0}: Error finding container 79e3c2c189fddd135fcaee4822e154305ade87fbe16bc1a58818548ec8394b9a: Status 404 returned error can't find the container with id 79e3c2c189fddd135fcaee4822e154305ade87fbe16bc1a58818548ec8394b9a
Jul 27 17:26:46.191481 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:46.191445    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7a4ca3-159a-4f46-9d85-a512339877e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-9kg5s\" (UID: \"ed7a4ca3-159a-4f46-9d85-a512339877e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:46.194025 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:46.193998    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7a4ca3-159a-4f46-9d85-a512339877e5-tls-certs\") pod \"model-serving-api-54d89bd5fc-9kg5s\" (UID: \"ed7a4ca3-159a-4f46-9d85-a512339877e5\") " pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:46.457807 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:46.457708    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:46.600910 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:46.600887    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9kg5s"]
Jul 27 17:26:46.603765 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:26:46.603734    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7a4ca3_159a_4f46_9d85_a512339877e5.slice/crio-08dc2de246f20593ff6d8e676cd8adf14af855e12da8eee661919db7658fa6af WatchSource:0}: Error finding container 08dc2de246f20593ff6d8e676cd8adf14af855e12da8eee661919db7658fa6af: Status 404 returned error can't find the container with id 08dc2de246f20593ff6d8e676cd8adf14af855e12da8eee661919db7658fa6af
Jul 27 17:26:46.757713 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:46.757606    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s" event={"ID":"ed7a4ca3-159a-4f46-9d85-a512339877e5","Type":"ContainerStarted","Data":"08dc2de246f20593ff6d8e676cd8adf14af855e12da8eee661919db7658fa6af"}
Jul 27 17:26:46.759085 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:46.759055    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx" event={"ID":"821bb5e4-886a-4df1-8c7b-1399e7da2bb3","Type":"ContainerStarted","Data":"79e3c2c189fddd135fcaee4822e154305ade87fbe16bc1a58818548ec8394b9a"}
Jul 27 17:26:47.519795 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.519670    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"]
Jul 27 17:26:47.523988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.523964    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.529058 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.529015    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 17:26:47.530915 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.530474    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:26:47.530915 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.530775    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 17:26:47.530915 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.530800    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-np22g\""
Jul 27 17:26:47.531714 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.530981    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 17:26:47.531714 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.531000    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 17:26:47.535131 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.534852    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"]
Jul 27 17:26:47.604994 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.604957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/943dd6cb-3abc-4725-b364-af87695556bb-manager-config\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.605502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.605059    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/943dd6cb-3abc-4725-b364-af87695556bb-metrics-cert\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.605502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.605094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943dd6cb-3abc-4725-b364-af87695556bb-cert\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.605502 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.605134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztnbn\" (UniqueName: \"kubernetes.io/projected/943dd6cb-3abc-4725-b364-af87695556bb-kube-api-access-ztnbn\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.706188 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.706145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/943dd6cb-3abc-4725-b364-af87695556bb-metrics-cert\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.706383 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.706200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943dd6cb-3abc-4725-b364-af87695556bb-cert\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.706383 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.706237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztnbn\" (UniqueName: \"kubernetes.io/projected/943dd6cb-3abc-4725-b364-af87695556bb-kube-api-access-ztnbn\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.706383 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.706276    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/943dd6cb-3abc-4725-b364-af87695556bb-manager-config\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.707076 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.707054    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/943dd6cb-3abc-4725-b364-af87695556bb-manager-config\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.712357 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.710431    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/943dd6cb-3abc-4725-b364-af87695556bb-metrics-cert\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.713310 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.713247    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/943dd6cb-3abc-4725-b364-af87695556bb-cert\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.737206 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.737172    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztnbn\" (UniqueName: \"kubernetes.io/projected/943dd6cb-3abc-4725-b364-af87695556bb-kube-api-access-ztnbn\") pod \"lws-controller-manager-76849bb8b4-wdrzp\" (UID: \"943dd6cb-3abc-4725-b364-af87695556bb\") " pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:47.839545 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:47.839507    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:50.163500 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.163466    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"]
Jul 27 17:26:50.167334 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:26:50.167307    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943dd6cb_3abc_4725_b364_af87695556bb.slice/crio-3de28f177b5416a704a457e43e900ed4def13eeddf4c3fe1cdd314883d8e6177 WatchSource:0}: Error finding container 3de28f177b5416a704a457e43e900ed4def13eeddf4c3fe1cdd314883d8e6177: Status 404 returned error can't find the container with id 3de28f177b5416a704a457e43e900ed4def13eeddf4c3fe1cdd314883d8e6177
Jul 27 17:26:50.782893 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.782854    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s" event={"ID":"ed7a4ca3-159a-4f46-9d85-a512339877e5","Type":"ContainerStarted","Data":"ff15e1ec0eef66648739fb7615684fa773537160606b3ef792bdeef52dec3c4f"}
Jul 27 17:26:50.783091 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.782921    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:26:50.784097 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.784067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp" event={"ID":"943dd6cb-3abc-4725-b364-af87695556bb","Type":"ContainerStarted","Data":"3de28f177b5416a704a457e43e900ed4def13eeddf4c3fe1cdd314883d8e6177"}
Jul 27 17:26:50.785478 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.785452    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s" event={"ID":"00da9703-3511-4e59-8048-72e5cd010f14","Type":"ContainerStarted","Data":"144a8f7d5a742452aec70692a74b08179f95e57ac141f4b4ee3ae296e2b7451f"}
Jul 27 17:26:50.786913 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.786889    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx" event={"ID":"821bb5e4-886a-4df1-8c7b-1399e7da2bb3","Type":"ContainerStarted","Data":"d7bb7dbc6b91e8f72605a81714d4b05d709fd0f817976c74342f7a0857f7b2ff"}
Jul 27 17:26:50.787034 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.787014    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:26:50.802539 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.802471    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s" podStartSLOduration=2.405505926 podStartE2EDuration="5.802454788s" totalImagesPullingTime="3.396948862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:50.80125326 +0000 UTC m=+368.062228459" watchObservedRunningTime="2026-07-27 17:26:50.802454788 +0000 UTC m=+368.063429965"
Jul 27 17:26:50.820380 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.820287    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx" podStartSLOduration=2.583486794 podStartE2EDuration="6.820269305s" totalImagesPullingTime="4.236782511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:50.8190401 +0000 UTC m=+368.080015299" watchObservedRunningTime="2026-07-27 17:26:50.820269305 +0000 UTC m=+368.081244505"
Jul 27 17:26:50.838629 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:50.838582    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b8c977c58-q975s" podStartSLOduration=1.843879643 podStartE2EDuration="6.83856651s" totalImagesPullingTime="4.994686867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:50.838043001 +0000 UTC m=+368.099018203" watchObservedRunningTime="2026-07-27 17:26:50.83856651 +0000 UTC m=+368.099541741"
Jul 27 17:26:53.805854 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:53.805818    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp" event={"ID":"943dd6cb-3abc-4725-b364-af87695556bb","Type":"ContainerStarted","Data":"30d26da95cb8eeca70c244753f495d77d7379f9bbedfdcb3004f70617a10d985"}
Jul 27 17:26:53.806247 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:53.805933    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:26:53.840106 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:26:53.840053    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp" podStartSLOduration=3.9150186160000002 podStartE2EDuration="6.840037971s" totalImagesPullingTime="2.925019355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:26:53.838624293 +0000 UTC m=+371.099599493" watchObservedRunningTime="2026-07-27 17:26:53.840037971 +0000 UTC m=+371.101013170"
Jul 27 17:27:01.796819 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:27:01.796788    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pjkkx"
Jul 27 17:27:01.798727 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:27:01.798706    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9kg5s"
Jul 27 17:27:04.812339 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:27:04.812283    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76849bb8b4-wdrzp"
Jul 27 17:29:25.597650 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.597616    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:29:25.599845 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.599826    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:25.602701 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.602682    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 17:29:25.603878 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.603856    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 17:29:25.603878 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.603871    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 17:29:25.604040 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.603896    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-scg5l\""
Jul 27 17:29:25.609203 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.609182    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:29:25.624109 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.624084    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:29:25.677767 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.677744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57986\" (UniqueName: \"kubernetes.io/projected/efc57564-8fdc-45b9-8f97-82d3f6de4aeb-kube-api-access-57986\") pod \"limitador-limitador-54d6d988d5-pdhh9\" (UID: \"efc57564-8fdc-45b9-8f97-82d3f6de4aeb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:25.677898 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.677784    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/efc57564-8fdc-45b9-8f97-82d3f6de4aeb-config-file\") pod \"limitador-limitador-54d6d988d5-pdhh9\" (UID: \"efc57564-8fdc-45b9-8f97-82d3f6de4aeb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:25.778590 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.778561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57986\" (UniqueName: \"kubernetes.io/projected/efc57564-8fdc-45b9-8f97-82d3f6de4aeb-kube-api-access-57986\") pod \"limitador-limitador-54d6d988d5-pdhh9\" (UID: \"efc57564-8fdc-45b9-8f97-82d3f6de4aeb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:25.778727 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.778605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/efc57564-8fdc-45b9-8f97-82d3f6de4aeb-config-file\") pod \"limitador-limitador-54d6d988d5-pdhh9\" (UID: \"efc57564-8fdc-45b9-8f97-82d3f6de4aeb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:25.779228 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.779209    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/efc57564-8fdc-45b9-8f97-82d3f6de4aeb-config-file\") pod \"limitador-limitador-54d6d988d5-pdhh9\" (UID: \"efc57564-8fdc-45b9-8f97-82d3f6de4aeb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:25.787708 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.787686    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57986\" (UniqueName: \"kubernetes.io/projected/efc57564-8fdc-45b9-8f97-82d3f6de4aeb-kube-api-access-57986\") pod \"limitador-limitador-54d6d988d5-pdhh9\" (UID: \"efc57564-8fdc-45b9-8f97-82d3f6de4aeb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:25.910612 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:25.910533    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:26.037175 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.037109    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:29:26.039661 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:29:26.039634    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc57564_8fdc_45b9_8f97_82d3f6de4aeb.slice/crio-9a25c6d84999b718377d4d29cd43a8c2838a1dad8dc6c18c5d9d9ad59ed8aa20 WatchSource:0}: Error finding container 9a25c6d84999b718377d4d29cd43a8c2838a1dad8dc6c18c5d9d9ad59ed8aa20: Status 404 returned error can't find the container with id 9a25c6d84999b718377d4d29cd43a8c2838a1dad8dc6c18c5d9d9ad59ed8aa20
Jul 27 17:29:26.072606 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.072576    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-24tnl"]
Jul 27 17:29:26.075301 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.075275    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-24tnl"
Jul 27 17:29:26.078999 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.078971    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-428tm\""
Jul 27 17:29:26.083220 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.083193    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-24tnl"]
Jul 27 17:29:26.182182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.182096    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k9fj\" (UniqueName: \"kubernetes.io/projected/3077eeab-3ec1-4719-b028-32e918c3876d-kube-api-access-7k9fj\") pod \"authorino-58cd955675-24tnl\" (UID: \"3077eeab-3ec1-4719-b028-32e918c3876d\") " pod="kuadrant-system/authorino-58cd955675-24tnl"
Jul 27 17:29:26.282519 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.282484    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k9fj\" (UniqueName: \"kubernetes.io/projected/3077eeab-3ec1-4719-b028-32e918c3876d-kube-api-access-7k9fj\") pod \"authorino-58cd955675-24tnl\" (UID: \"3077eeab-3ec1-4719-b028-32e918c3876d\") " pod="kuadrant-system/authorino-58cd955675-24tnl"
Jul 27 17:29:26.290856 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.290826    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k9fj\" (UniqueName: \"kubernetes.io/projected/3077eeab-3ec1-4719-b028-32e918c3876d-kube-api-access-7k9fj\") pod \"authorino-58cd955675-24tnl\" (UID: \"3077eeab-3ec1-4719-b028-32e918c3876d\") " pod="kuadrant-system/authorino-58cd955675-24tnl"
Jul 27 17:29:26.384672 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.384640    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-24tnl"
Jul 27 17:29:26.456822 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.456788    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9" event={"ID":"efc57564-8fdc-45b9-8f97-82d3f6de4aeb","Type":"ContainerStarted","Data":"9a25c6d84999b718377d4d29cd43a8c2838a1dad8dc6c18c5d9d9ad59ed8aa20"}
Jul 27 17:29:26.518069 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:26.518038    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-24tnl"]
Jul 27 17:29:26.519477 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:29:26.519434    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3077eeab_3ec1_4719_b028_32e918c3876d.slice/crio-34a24af0a1ed489e7c1cd7dd3e23d7907d875ea90091f49a50d1e6bfa3ff2cdb WatchSource:0}: Error finding container 34a24af0a1ed489e7c1cd7dd3e23d7907d875ea90091f49a50d1e6bfa3ff2cdb: Status 404 returned error can't find the container with id 34a24af0a1ed489e7c1cd7dd3e23d7907d875ea90091f49a50d1e6bfa3ff2cdb
Jul 27 17:29:27.463664 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:27.463621    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-24tnl" event={"ID":"3077eeab-3ec1-4719-b028-32e918c3876d","Type":"ContainerStarted","Data":"34a24af0a1ed489e7c1cd7dd3e23d7907d875ea90091f49a50d1e6bfa3ff2cdb"}
Jul 27 17:29:32.285983 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:32.285944    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-24tnl"]
Jul 27 17:29:32.490047 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:32.490007    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-24tnl" event={"ID":"3077eeab-3ec1-4719-b028-32e918c3876d","Type":"ContainerStarted","Data":"d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e"}
Jul 27 17:29:32.491501 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:32.491474    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9" event={"ID":"efc57564-8fdc-45b9-8f97-82d3f6de4aeb","Type":"ContainerStarted","Data":"ab176f0cc4fcf76d127468fa9eddc468402e7aa6dbd9c2cbf543fb6b708017bd"}
Jul 27 17:29:32.491651 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:32.491605    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:32.507280 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:32.507236    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-24tnl" podStartSLOduration=1.5782135880000001 podStartE2EDuration="6.507222789s" totalImagesPullingTime="4.929009201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:32.506261169 +0000 UTC m=+529.767236572" watchObservedRunningTime="2026-07-27 17:29:32.507222789 +0000 UTC m=+529.768197988"
Jul 27 17:29:32.527959 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:32.527909    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9" podStartSLOduration=2.066269583 podStartE2EDuration="7.527895216s" totalImagesPullingTime="5.461625633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:32.526464146 +0000 UTC m=+529.787439344" watchObservedRunningTime="2026-07-27 17:29:32.527895216 +0000 UTC m=+529.788870414"
Jul 27 17:29:33.495270 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:33.495225    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-24tnl" podUID="3077eeab-3ec1-4719-b028-32e918c3876d" containerName="authorino" containerID="cri-o://d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e" gracePeriod=30
Jul 27 17:29:33.738760 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:33.738731    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-24tnl"
Jul 27 17:29:33.848512 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:33.848480    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k9fj\" (UniqueName: \"kubernetes.io/projected/3077eeab-3ec1-4719-b028-32e918c3876d-kube-api-access-7k9fj\") pod \"3077eeab-3ec1-4719-b028-32e918c3876d\" (UID: \"3077eeab-3ec1-4719-b028-32e918c3876d\") "
Jul 27 17:29:33.850655 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:33.850633    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3077eeab-3ec1-4719-b028-32e918c3876d-kube-api-access-7k9fj" (OuterVolumeSpecName: "kube-api-access-7k9fj") pod "3077eeab-3ec1-4719-b028-32e918c3876d" (UID: "3077eeab-3ec1-4719-b028-32e918c3876d"). InnerVolumeSpecName "kube-api-access-7k9fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:29:33.949852 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:33.949812    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k9fj\" (UniqueName: \"kubernetes.io/projected/3077eeab-3ec1-4719-b028-32e918c3876d-kube-api-access-7k9fj\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:29:34.500809 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.500769    2565 generic.go:356] "Generic (PLEG): container finished" podID="3077eeab-3ec1-4719-b028-32e918c3876d" containerID="d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e" exitCode=0
Jul 27 17:29:34.500809 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.500809    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-24tnl"
Jul 27 17:29:34.501375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.500837    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-24tnl" event={"ID":"3077eeab-3ec1-4719-b028-32e918c3876d","Type":"ContainerDied","Data":"d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e"}
Jul 27 17:29:34.501375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.500869    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-24tnl" event={"ID":"3077eeab-3ec1-4719-b028-32e918c3876d","Type":"ContainerDied","Data":"34a24af0a1ed489e7c1cd7dd3e23d7907d875ea90091f49a50d1e6bfa3ff2cdb"}
Jul 27 17:29:34.501375 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.500885    2565 scope.go:117] "RemoveContainer" containerID="d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e"
Jul 27 17:29:34.511735 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.510946    2565 scope.go:117] "RemoveContainer" containerID="d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e"
Jul 27 17:29:34.512195 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:29:34.512175    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e\": container with ID starting with d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e not found: ID does not exist" containerID="d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e"
Jul 27 17:29:34.512239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.512205    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e"} err="failed to get container status \"d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e\": rpc error: code = NotFound desc = could not find container \"d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e\": container with ID starting with d8ab24f621cdf4bbbfc4ddf11d6e794ee49e1fcd3b9858beff863b32825a662e not found: ID does not exist"
Jul 27 17:29:34.522612 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.522590    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-24tnl"]
Jul 27 17:29:34.525166 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:34.525145    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-24tnl"]
Jul 27 17:29:35.255397 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:35.255285    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3077eeab-3ec1-4719-b028-32e918c3876d" path="/var/lib/kubelet/pods/3077eeab-3ec1-4719-b028-32e918c3876d/volumes"
Jul 27 17:29:43.498181 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:43.498090    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"
Jul 27 17:29:58.903670 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.903636    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zb2m"]
Jul 27 17:29:58.904157 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.903982    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3077eeab-3ec1-4719-b028-32e918c3876d" containerName="authorino"
Jul 27 17:29:58.904157 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.903995    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3077eeab-3ec1-4719-b028-32e918c3876d" containerName="authorino"
Jul 27 17:29:58.904157 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.904052    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3077eeab-3ec1-4719-b028-32e918c3876d" containerName="authorino"
Jul 27 17:29:58.908066 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.908035    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zb2m"
Jul 27 17:29:58.910904 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.910884    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-428tm\""
Jul 27 17:29:58.914077 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.914053    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zb2m"]
Jul 27 17:29:58.943082 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:58.943044    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgcml\" (UniqueName: \"kubernetes.io/projected/6c87bba8-679a-4ccd-a7ef-c5848c8d6970-kube-api-access-kgcml\") pod \"authorino-5d4b67c446-7zb2m\" (UID: \"6c87bba8-679a-4ccd-a7ef-c5848c8d6970\") " pod="kuadrant-system/authorino-5d4b67c446-7zb2m"
Jul 27 17:29:59.043856 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.043818    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgcml\" (UniqueName: \"kubernetes.io/projected/6c87bba8-679a-4ccd-a7ef-c5848c8d6970-kube-api-access-kgcml\") pod \"authorino-5d4b67c446-7zb2m\" (UID: \"6c87bba8-679a-4ccd-a7ef-c5848c8d6970\") " pod="kuadrant-system/authorino-5d4b67c446-7zb2m"
Jul 27 17:29:59.052362 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.052332    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgcml\" (UniqueName: \"kubernetes.io/projected/6c87bba8-679a-4ccd-a7ef-c5848c8d6970-kube-api-access-kgcml\") pod \"authorino-5d4b67c446-7zb2m\" (UID: \"6c87bba8-679a-4ccd-a7ef-c5848c8d6970\") " pod="kuadrant-system/authorino-5d4b67c446-7zb2m"
Jul 27 17:29:59.109729 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.109699    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zb2m"]
Jul 27 17:29:59.109952 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.109940    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zb2m"
Jul 27 17:29:59.134368 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.134339    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6746894fdd-pvmnn"]
Jul 27 17:29:59.138107 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.138087    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.141182 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.141155    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 27 17:29:59.144522 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.144489    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4571aac2-84ff-4fe7-a518-2fea202cc270-tls-cert\") pod \"authorino-6746894fdd-pvmnn\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") " pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.144700 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.144560    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhxlr\" (UniqueName: \"kubernetes.io/projected/4571aac2-84ff-4fe7-a518-2fea202cc270-kube-api-access-vhxlr\") pod \"authorino-6746894fdd-pvmnn\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") " pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.148216 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.148195    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6746894fdd-pvmnn"]
Jul 27 17:29:59.238287 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.238257    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zb2m"]
Jul 27 17:29:59.240498 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:29:59.240473    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c87bba8_679a_4ccd_a7ef_c5848c8d6970.slice/crio-7cd0abeb478a8f34462e49987484dc1169cafa2f86e88b9268cbe800d95584cd WatchSource:0}: Error finding container 7cd0abeb478a8f34462e49987484dc1169cafa2f86e88b9268cbe800d95584cd: Status 404 returned error can't find the container with id 7cd0abeb478a8f34462e49987484dc1169cafa2f86e88b9268cbe800d95584cd
Jul 27 17:29:59.245722 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.245698    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4571aac2-84ff-4fe7-a518-2fea202cc270-tls-cert\") pod \"authorino-6746894fdd-pvmnn\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") " pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.245819 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.245744    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhxlr\" (UniqueName: \"kubernetes.io/projected/4571aac2-84ff-4fe7-a518-2fea202cc270-kube-api-access-vhxlr\") pod \"authorino-6746894fdd-pvmnn\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") " pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.248277 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.248258    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4571aac2-84ff-4fe7-a518-2fea202cc270-tls-cert\") pod \"authorino-6746894fdd-pvmnn\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") " pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.255169 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.255146    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhxlr\" (UniqueName: \"kubernetes.io/projected/4571aac2-84ff-4fe7-a518-2fea202cc270-kube-api-access-vhxlr\") pod \"authorino-6746894fdd-pvmnn\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") " pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.447745 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.447654    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:29:59.573699 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.573669    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6746894fdd-pvmnn"]
Jul 27 17:29:59.575792 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:29:59.575767    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4571aac2_84ff_4fe7_a518_2fea202cc270.slice/crio-71c7b9977682407ba9ff94efd13be5748d9c1ddafdac4b3e9c109ee7d8553686 WatchSource:0}: Error finding container 71c7b9977682407ba9ff94efd13be5748d9c1ddafdac4b3e9c109ee7d8553686: Status 404 returned error can't find the container with id 71c7b9977682407ba9ff94efd13be5748d9c1ddafdac4b3e9c109ee7d8553686
Jul 27 17:29:59.607786 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.607748    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6746894fdd-pvmnn" event={"ID":"4571aac2-84ff-4fe7-a518-2fea202cc270","Type":"ContainerStarted","Data":"71c7b9977682407ba9ff94efd13be5748d9c1ddafdac4b3e9c109ee7d8553686"}
Jul 27 17:29:59.608852 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:29:59.608831    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zb2m" event={"ID":"6c87bba8-679a-4ccd-a7ef-c5848c8d6970","Type":"ContainerStarted","Data":"7cd0abeb478a8f34462e49987484dc1169cafa2f86e88b9268cbe800d95584cd"}
Jul 27 17:30:00.615027 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.614986    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zb2m" event={"ID":"6c87bba8-679a-4ccd-a7ef-c5848c8d6970","Type":"ContainerStarted","Data":"6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc"}
Jul 27 17:30:00.615027 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.614994    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7zb2m" podUID="6c87bba8-679a-4ccd-a7ef-c5848c8d6970" containerName="authorino" containerID="cri-o://6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc" gracePeriod=30
Jul 27 17:30:00.616613 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.616582    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6746894fdd-pvmnn" event={"ID":"4571aac2-84ff-4fe7-a518-2fea202cc270","Type":"ContainerStarted","Data":"c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a"}
Jul 27 17:30:00.632582 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.632540    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7zb2m" podStartSLOduration=2.14568507 podStartE2EDuration="2.632526804s" totalImagesPullingTime="486.841734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:30:00.631142286 +0000 UTC m=+557.892117488" watchObservedRunningTime="2026-07-27 17:30:00.632526804 +0000 UTC m=+557.893502004"
Jul 27 17:30:00.648482 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.648438    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6746894fdd-pvmnn" podStartSLOduration=1.259484 podStartE2EDuration="1.648424352s" totalImagesPullingTime="388.940352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:30:00.647256885 +0000 UTC m=+557.908232087" watchObservedRunningTime="2026-07-27 17:30:00.648424352 +0000 UTC m=+557.909399550"
Jul 27 17:30:00.859068 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.859042    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zb2m"
Jul 27 17:30:00.960056 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.959979    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgcml\" (UniqueName: \"kubernetes.io/projected/6c87bba8-679a-4ccd-a7ef-c5848c8d6970-kube-api-access-kgcml\") pod \"6c87bba8-679a-4ccd-a7ef-c5848c8d6970\" (UID: \"6c87bba8-679a-4ccd-a7ef-c5848c8d6970\") "
Jul 27 17:30:00.961982 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:00.961955    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c87bba8-679a-4ccd-a7ef-c5848c8d6970-kube-api-access-kgcml" (OuterVolumeSpecName: "kube-api-access-kgcml") pod "6c87bba8-679a-4ccd-a7ef-c5848c8d6970" (UID: "6c87bba8-679a-4ccd-a7ef-c5848c8d6970"). InnerVolumeSpecName "kube-api-access-kgcml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:30:01.060711 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.060678    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgcml\" (UniqueName: \"kubernetes.io/projected/6c87bba8-679a-4ccd-a7ef-c5848c8d6970-kube-api-access-kgcml\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:30:01.623172 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.623134    2565 generic.go:356] "Generic (PLEG): container finished" podID="6c87bba8-679a-4ccd-a7ef-c5848c8d6970" containerID="6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc" exitCode=0
Jul 27 17:30:01.623680 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.623195    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zb2m"
Jul 27 17:30:01.623680 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.623222    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zb2m" event={"ID":"6c87bba8-679a-4ccd-a7ef-c5848c8d6970","Type":"ContainerDied","Data":"6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc"}
Jul 27 17:30:01.623680 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.623264    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zb2m" event={"ID":"6c87bba8-679a-4ccd-a7ef-c5848c8d6970","Type":"ContainerDied","Data":"7cd0abeb478a8f34462e49987484dc1169cafa2f86e88b9268cbe800d95584cd"}
Jul 27 17:30:01.623680 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.623282    2565 scope.go:117] "RemoveContainer" containerID="6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc"
Jul 27 17:30:01.631343 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.631327    2565 scope.go:117] "RemoveContainer" containerID="6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc"
Jul 27 17:30:01.631585 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:30:01.631567    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc\": container with ID starting with 6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc not found: ID does not exist" containerID="6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc"
Jul 27 17:30:01.631647 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.631592    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc"} err="failed to get container status \"6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc\": rpc error: code = NotFound desc = could not find container \"6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc\": container with ID starting with 6788a31fc2b80613fadd6c0090a78b7f818bb9f0d555c18f26e163f5d28b7dcc not found: ID does not exist"
Jul 27 17:30:01.646133 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.646105    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zb2m"]
Jul 27 17:30:01.650594 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:01.650566    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zb2m"]
Jul 27 17:30:03.255879 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:03.255845    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c87bba8-679a-4ccd-a7ef-c5848c8d6970" path="/var/lib/kubelet/pods/6c87bba8-679a-4ccd-a7ef-c5848c8d6970/volumes"
Jul 27 17:30:42.977942 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:42.977899    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:30:43.146925 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:43.146900    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:30:43.150106 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:30:43.150081    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:31:00.084930 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:31:00.084889    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:31:07.107986 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:31:07.107906    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:31:13.875154 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:31:13.875114    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:31:36.075309 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:31:36.075251    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:31:46.280354 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:31:46.280320    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:32:16.877153 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.877108    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"]
Jul 27 17:32:16.877671 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.877475    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c87bba8-679a-4ccd-a7ef-c5848c8d6970" containerName="authorino"
Jul 27 17:32:16.877671 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.877487    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c87bba8-679a-4ccd-a7ef-c5848c8d6970" containerName="authorino"
Jul 27 17:32:16.877671 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.877543    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c87bba8-679a-4ccd-a7ef-c5848c8d6970" containerName="authorino"
Jul 27 17:32:16.879274 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.879255    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:16.887779 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.887755    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 27 17:32:16.887922 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.887859    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zlcs9\""
Jul 27 17:32:16.889457 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.889436    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 17:32:16.889560 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.889503    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 17:32:16.893212 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.893188    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"]
Jul 27 17:32:16.962044 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.962007    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6169d61-67d5-4c21-8fc9-e4638bee575d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:16.962228 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.962057    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:16.962228 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.962145    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:16.962228 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.962186    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:16.962228 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.962207    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:16.962388 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:16.962267    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbhhz\" (UniqueName: \"kubernetes.io/projected/c6169d61-67d5-4c21-8fc9-e4638bee575d-kube-api-access-wbhhz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063228 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063434 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063261    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063434 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063434 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063341    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063434 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbhhz\" (UniqueName: \"kubernetes.io/projected/c6169d61-67d5-4c21-8fc9-e4638bee575d-kube-api-access-wbhhz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063434 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063418    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6169d61-67d5-4c21-8fc9-e4638bee575d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063658 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063636    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063709 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.063779 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.063755    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.065536 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.065515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6169d61-67d5-4c21-8fc9-e4638bee575d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.065833 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.065816    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6169d61-67d5-4c21-8fc9-e4638bee575d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.074381 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.074350    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbhhz\" (UniqueName: \"kubernetes.io/projected/c6169d61-67d5-4c21-8fc9-e4638bee575d-kube-api-access-wbhhz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq\" (UID: \"c6169d61-67d5-4c21-8fc9-e4638bee575d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.197912 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.197818    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:17.333043 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.333014    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"]
Jul 27 17:32:17.334493 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:32:17.334464    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6169d61_67d5_4c21_8fc9_e4638bee575d.slice/crio-8a5d10fbf811f275b376cd007fe9ff98736c4a18c7803a8c42cd8c267e760b3f WatchSource:0}: Error finding container 8a5d10fbf811f275b376cd007fe9ff98736c4a18c7803a8c42cd8c267e760b3f: Status 404 returned error can't find the container with id 8a5d10fbf811f275b376cd007fe9ff98736c4a18c7803a8c42cd8c267e760b3f
Jul 27 17:32:17.335836 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:17.335820    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:32:18.230628 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:18.230591    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq" event={"ID":"c6169d61-67d5-4c21-8fc9-e4638bee575d","Type":"ContainerStarted","Data":"8a5d10fbf811f275b376cd007fe9ff98736c4a18c7803a8c42cd8c267e760b3f"}
Jul 27 17:32:20.274871 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:20.274837    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:32:26.271783 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:26.271745    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq" event={"ID":"c6169d61-67d5-4c21-8fc9-e4638bee575d","Type":"ContainerStarted","Data":"e45d644273d8941c114a37e258a64d8b928fc4f454d395b87a9974fe362f356a"}
Jul 27 17:32:26.292864 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:26.292817    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq" podStartSLOduration=1.835989946 podStartE2EDuration="10.292802808s" totalImagesPullingTime="8.456812862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:26.292074263 +0000 UTC m=+703.553049465" watchObservedRunningTime="2026-07-27 17:32:26.292802808 +0000 UTC m=+703.553778006"
Jul 27 17:32:27.198280 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:27.198238    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:27.198491 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:27.198312    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:27.210768 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:27.210737    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:32:27.286725 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:32:27.286695    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq"
Jul 27 17:33:18.513236 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.513197    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67dbb79db9-bqr6h"]
Jul 27 17:33:18.516822 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.516802    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.527450 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.527426    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67dbb79db9-bqr6h"]
Jul 27 17:33:18.575853 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.575825    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5111231d-9b81-4cb9-9f44-0799c34432e9-tls-cert\") pod \"authorino-67dbb79db9-bqr6h\" (UID: \"5111231d-9b81-4cb9-9f44-0799c34432e9\") " pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.575976 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.575858    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s96b\" (UniqueName: \"kubernetes.io/projected/5111231d-9b81-4cb9-9f44-0799c34432e9-kube-api-access-6s96b\") pod \"authorino-67dbb79db9-bqr6h\" (UID: \"5111231d-9b81-4cb9-9f44-0799c34432e9\") " pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.677273 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.677244    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5111231d-9b81-4cb9-9f44-0799c34432e9-tls-cert\") pod \"authorino-67dbb79db9-bqr6h\" (UID: \"5111231d-9b81-4cb9-9f44-0799c34432e9\") " pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.677422 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.677281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s96b\" (UniqueName: \"kubernetes.io/projected/5111231d-9b81-4cb9-9f44-0799c34432e9-kube-api-access-6s96b\") pod \"authorino-67dbb79db9-bqr6h\" (UID: \"5111231d-9b81-4cb9-9f44-0799c34432e9\") " pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.679752 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.679730    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5111231d-9b81-4cb9-9f44-0799c34432e9-tls-cert\") pod \"authorino-67dbb79db9-bqr6h\" (UID: \"5111231d-9b81-4cb9-9f44-0799c34432e9\") " pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.686707 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.686685    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s96b\" (UniqueName: \"kubernetes.io/projected/5111231d-9b81-4cb9-9f44-0799c34432e9-kube-api-access-6s96b\") pod \"authorino-67dbb79db9-bqr6h\" (UID: \"5111231d-9b81-4cb9-9f44-0799c34432e9\") " pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.825848 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.825819    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67dbb79db9-bqr6h"
Jul 27 17:33:18.963441 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:18.963415    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67dbb79db9-bqr6h"]
Jul 27 17:33:18.965390 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:33:18.965361    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5111231d_9b81_4cb9_9f44_0799c34432e9.slice/crio-88ed2e721f843825d98324363ffcac2455fad87d1869800fc7ccb4e24d73909a WatchSource:0}: Error finding container 88ed2e721f843825d98324363ffcac2455fad87d1869800fc7ccb4e24d73909a: Status 404 returned error can't find the container with id 88ed2e721f843825d98324363ffcac2455fad87d1869800fc7ccb4e24d73909a
Jul 27 17:33:19.498091 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:19.498054    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67dbb79db9-bqr6h" event={"ID":"5111231d-9b81-4cb9-9f44-0799c34432e9","Type":"ContainerStarted","Data":"88ed2e721f843825d98324363ffcac2455fad87d1869800fc7ccb4e24d73909a"}
Jul 27 17:33:20.504721 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:20.504680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67dbb79db9-bqr6h" event={"ID":"5111231d-9b81-4cb9-9f44-0799c34432e9","Type":"ContainerStarted","Data":"c8e6a057053b683d7fb1c22885b7f6692daa921f2b5d2d9b0837d1fe4c32ff84"}
Jul 27 17:33:20.534986 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:20.534918    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67dbb79db9-bqr6h" podStartSLOduration=1.721289635 podStartE2EDuration="2.534897781s" totalImagesPullingTime="813.608146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:20.533183929 +0000 UTC m=+757.794159130" watchObservedRunningTime="2026-07-27 17:33:20.534897781 +0000 UTC m=+757.795872987"
Jul 27 17:33:20.623206 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:20.623169    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6746894fdd-pvmnn"]
Jul 27 17:33:20.623983 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:20.623698    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6746894fdd-pvmnn" podUID="4571aac2-84ff-4fe7-a518-2fea202cc270" containerName="authorino" containerID="cri-o://c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a" gracePeriod=30
Jul 27 17:33:20.886047 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:20.886017    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:33:20.999319 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:20.998544    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4571aac2-84ff-4fe7-a518-2fea202cc270-tls-cert\") pod \"4571aac2-84ff-4fe7-a518-2fea202cc270\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") "
Jul 27 17:33:20.999319 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:20.998669    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhxlr\" (UniqueName: \"kubernetes.io/projected/4571aac2-84ff-4fe7-a518-2fea202cc270-kube-api-access-vhxlr\") pod \"4571aac2-84ff-4fe7-a518-2fea202cc270\" (UID: \"4571aac2-84ff-4fe7-a518-2fea202cc270\") "
Jul 27 17:33:21.012481 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.010523    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4571aac2-84ff-4fe7-a518-2fea202cc270-kube-api-access-vhxlr" (OuterVolumeSpecName: "kube-api-access-vhxlr") pod "4571aac2-84ff-4fe7-a518-2fea202cc270" (UID: "4571aac2-84ff-4fe7-a518-2fea202cc270"). InnerVolumeSpecName "kube-api-access-vhxlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:33:21.023045 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.023011    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4571aac2-84ff-4fe7-a518-2fea202cc270-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4571aac2-84ff-4fe7-a518-2fea202cc270" (UID: "4571aac2-84ff-4fe7-a518-2fea202cc270"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:33:21.100020 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.099993    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhxlr\" (UniqueName: \"kubernetes.io/projected/4571aac2-84ff-4fe7-a518-2fea202cc270-kube-api-access-vhxlr\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:33:21.100020 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.100017    2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4571aac2-84ff-4fe7-a518-2fea202cc270-tls-cert\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:33:21.510229 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.510137    2565 generic.go:356] "Generic (PLEG): container finished" podID="4571aac2-84ff-4fe7-a518-2fea202cc270" containerID="c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a" exitCode=0
Jul 27 17:33:21.510229 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.510182    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6746894fdd-pvmnn"
Jul 27 17:33:21.510229 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.510218    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6746894fdd-pvmnn" event={"ID":"4571aac2-84ff-4fe7-a518-2fea202cc270","Type":"ContainerDied","Data":"c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a"}
Jul 27 17:33:21.510784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.510265    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6746894fdd-pvmnn" event={"ID":"4571aac2-84ff-4fe7-a518-2fea202cc270","Type":"ContainerDied","Data":"71c7b9977682407ba9ff94efd13be5748d9c1ddafdac4b3e9c109ee7d8553686"}
Jul 27 17:33:21.510784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.510290    2565 scope.go:117] "RemoveContainer" containerID="c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a"
Jul 27 17:33:21.518961 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.518944    2565 scope.go:117] "RemoveContainer" containerID="c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a"
Jul 27 17:33:21.519234 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:33:21.519215    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a\": container with ID starting with c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a not found: ID does not exist" containerID="c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a"
Jul 27 17:33:21.519320 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.519240    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a"} err="failed to get container status \"c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a\": rpc error: code = NotFound desc = could not find container \"c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a\": container with ID starting with c1b2f69f735e89dad5e5d3109ab11db13ea6de0028af7d88b43fa2f9b22a7d7a not found: ID does not exist"
Jul 27 17:33:21.529780 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.529759    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6746894fdd-pvmnn"]
Jul 27 17:33:21.534320 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:21.534281    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6746894fdd-pvmnn"]
Jul 27 17:33:23.255638 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:23.255604    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4571aac2-84ff-4fe7-a518-2fea202cc270" path="/var/lib/kubelet/pods/4571aac2-84ff-4fe7-a518-2fea202cc270/volumes"
Jul 27 17:33:48.273410 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:48.273374    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:33:57.377941 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:33:57.375967    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:34:08.175769 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:34:08.175688    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:35:43.181504 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:35:43.181428    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:35:43.183563 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:35:43.183542    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:35:49.486722 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:35:49.486670    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:36:02.172833 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:36:02.172797    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:36:36.675428 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:36:36.675383    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:36:45.965281 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:36:45.965246    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:36:54.968069 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:36:54.968033    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:37:05.974112 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:37:05.974020    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:37:16.067323 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:37:16.067275    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:37:27.689230 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:37:27.689195    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:37:38.775438 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:37:38.775401    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:37:48.281310 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:37:48.281270    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:38:10.972117 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:38:10.972079    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:38:21.369313 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:38:21.369262    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:38:45.467543 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:38:45.467463    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:38:55.466884 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:38:55.466849    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:39:11.274795 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:39:11.274758    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:39:21.065027 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:39:21.064992    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:39:30.468288 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:39:30.468252    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:39:39.577704 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:39:39.577663    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:39:48.777073 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:39:48.777036    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:40:43.217349 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:40:43.217319    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:40:43.218166 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:40:43.218141    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:41:38.764875 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:41:38.764792    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:41:47.474835 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:41:47.474800    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:42:00.975620 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:42:00.975582    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:42:11.268575 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:42:11.268545    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:42:21.570989 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:42:21.570950    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:42:32.177037 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:42:32.176998    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:42:42.277832 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:42:42.277794    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:42:52.068384 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:42:52.068343    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:43:03.373701 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:43:03.373665    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:43:11.880412 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:43:11.880329    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:43:25.067360 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:43:25.067327    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:43:42.074116 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:43:42.074077    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:43:51.566878 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:43:51.566843    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:44:00.473736 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.473700    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"]
Jul 27 17:44:00.474218 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.474018    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4571aac2-84ff-4fe7-a518-2fea202cc270" containerName="authorino"
Jul 27 17:44:00.474218 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.474028    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4571aac2-84ff-4fe7-a518-2fea202cc270" containerName="authorino"
Jul 27 17:44:00.474218 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.474097    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4571aac2-84ff-4fe7-a518-2fea202cc270" containerName="authorino"
Jul 27 17:44:00.477001 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.476986    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.479785 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.479763    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-01b5eaab-kserve-self-signed-certs\""
Jul 27 17:44:00.495516 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.495492    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"]
Jul 27 17:44:00.587643 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.587613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxtgw\" (UniqueName: \"kubernetes.io/projected/c4be4456-1e6a-48d5-91b1-4bca6296d983-kube-api-access-wxtgw\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.587809 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.587663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-dshm\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.587809 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.587691    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-model-cache\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.587809 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.587755    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-tmp-dir\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.587809 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.587796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4be4456-1e6a-48d5-91b1-4bca6296d983-tls-certs\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.587954 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.587851    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-home\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.688581 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.688547    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-dshm\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.688581 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.688581    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-model-cache\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.688782 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.688603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-tmp-dir\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.688782 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.688623    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4be4456-1e6a-48d5-91b1-4bca6296d983-tls-certs\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.688782 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.688691    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-home\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.688782 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.688721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxtgw\" (UniqueName: \"kubernetes.io/projected/c4be4456-1e6a-48d5-91b1-4bca6296d983-kube-api-access-wxtgw\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.689101 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.689068    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-model-cache\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.689221 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.689120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-tmp-dir\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.689221 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.689168    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-home\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.690799 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.690782    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-dshm\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.691105 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.691088    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4be4456-1e6a-48d5-91b1-4bca6296d983-tls-certs\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.696798 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.696779    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxtgw\" (UniqueName: \"kubernetes.io/projected/c4be4456-1e6a-48d5-91b1-4bca6296d983-kube-api-access-wxtgw\") pod \"e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") " pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.786655 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.786574    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:00.913962 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:00.913935    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"]
Jul 27 17:44:00.915609 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:44:00.915582    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4be4456_1e6a_48d5_91b1_4bca6296d983.slice/crio-98c9d8ee5750337a7c91431a0ea786a224da84ecdbd5c49586b6e8bad571bd93 WatchSource:0}: Error finding container 98c9d8ee5750337a7c91431a0ea786a224da84ecdbd5c49586b6e8bad571bd93: Status 404 returned error can't find the container with id 98c9d8ee5750337a7c91431a0ea786a224da84ecdbd5c49586b6e8bad571bd93
Jul 27 17:44:01.393511 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:01.393470    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx" event={"ID":"c4be4456-1e6a-48d5-91b1-4bca6296d983","Type":"ContainerStarted","Data":"a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834"}
Jul 27 17:44:01.393693 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:01.393520    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx" event={"ID":"c4be4456-1e6a-48d5-91b1-4bca6296d983","Type":"ContainerStarted","Data":"98c9d8ee5750337a7c91431a0ea786a224da84ecdbd5c49586b6e8bad571bd93"}
Jul 27 17:44:01.411261 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:01.411212    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx" podStartSLOduration=1.411197443 podStartE2EDuration="1.411197443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:44:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:44:01.409756922 +0000 UTC m=+1398.670732121" watchObservedRunningTime="2026-07-27 17:44:01.411197443 +0000 UTC m=+1398.672172641"
Jul 27 17:44:03.274971 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:03.274924    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:44:10.457655 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.457618    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"]
Jul 27 17:44:10.461167 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.461147    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.464120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.464101    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-01b5eaab-kserve-self-signed-certs\""
Jul 27 17:44:10.464459 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.464432    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80cf6585-02b1-41a7-b624-286b00d8f6d7-tls-certs\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.464514 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.464500    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-home\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.464582 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.464566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfbkd\" (UniqueName: \"kubernetes.io/projected/80cf6585-02b1-41a7-b624-286b00d8f6d7-kube-api-access-rfbkd\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.464618 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.464599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-tmp-dir\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.464688 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.464674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-dshm\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.464735 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.464722    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-model-cache\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.471215 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.471192    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"]
Jul 27 17:44:10.565739 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.565709    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80cf6585-02b1-41a7-b624-286b00d8f6d7-tls-certs\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.565929 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.565748    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-home\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.565929 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.565782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfbkd\" (UniqueName: \"kubernetes.io/projected/80cf6585-02b1-41a7-b624-286b00d8f6d7-kube-api-access-rfbkd\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.565929 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.565800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-tmp-dir\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.565929 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.565842    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-dshm\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.565929 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.565864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-model-cache\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.566221 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.566175    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-home\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.566221 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.566203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-tmp-dir\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.566320 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.566221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-model-cache\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.568171 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.568143    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-dshm\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.568282 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.568219    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80cf6585-02b1-41a7-b624-286b00d8f6d7-tls-certs\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.575432 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.575407    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfbkd\" (UniqueName: \"kubernetes.io/projected/80cf6585-02b1-41a7-b624-286b00d8f6d7-kube-api-access-rfbkd\") pod \"e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") " pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.771958 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.771868    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:10.787004 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.786972    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:10.787126 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.787024    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:10.800751 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.800727    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:10.904332 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:10.904283    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"]
Jul 27 17:44:10.905640 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:44:10.905612    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80cf6585_02b1_41a7_b624_286b00d8f6d7.slice/crio-8779eec552bca7b87cdb15348901eb988d79fed4a102c5f2258bbb250e08a3d5 WatchSource:0}: Error finding container 8779eec552bca7b87cdb15348901eb988d79fed4a102c5f2258bbb250e08a3d5: Status 404 returned error can't find the container with id 8779eec552bca7b87cdb15348901eb988d79fed4a102c5f2258bbb250e08a3d5
Jul 27 17:44:11.441262 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:11.441222    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7" event={"ID":"80cf6585-02b1-41a7-b624-286b00d8f6d7","Type":"ContainerStarted","Data":"67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf"}
Jul 27 17:44:11.441454 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:11.441272    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7" event={"ID":"80cf6585-02b1-41a7-b624-286b00d8f6d7","Type":"ContainerStarted","Data":"8779eec552bca7b87cdb15348901eb988d79fed4a102c5f2258bbb250e08a3d5"}
Jul 27 17:44:11.452373 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:11.452352    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:11.459331 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:11.459276    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7" podStartSLOduration=1.459263123 podStartE2EDuration="1.459263123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:44:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:44:11.457860822 +0000 UTC m=+1408.718836021" watchObservedRunningTime="2026-07-27 17:44:11.459263123 +0000 UTC m=+1408.720238300"
Jul 27 17:44:15.183704 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:15.183667    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:44:20.772771 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:20.772733    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:20.773161 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:20.772799    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:20.785570 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:20.785540    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:21.497500 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:21.497470    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:28.734908 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:28.734875    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"]
Jul 27 17:44:29.871722 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:29.871689    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"]
Jul 27 17:44:32.863196 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:32.863165    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:44:43.754007 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:43.753917    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx" podUID="c4be4456-1e6a-48d5-91b1-4bca6296d983" containerName="main" containerID="cri-o://a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834" gracePeriod=45
Jul 27 17:44:43.999739 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:43.999715    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:44.129395 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129363    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxtgw\" (UniqueName: \"kubernetes.io/projected/c4be4456-1e6a-48d5-91b1-4bca6296d983-kube-api-access-wxtgw\") pod \"c4be4456-1e6a-48d5-91b1-4bca6296d983\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") "
Jul 27 17:44:44.129395 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129399    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-home\") pod \"c4be4456-1e6a-48d5-91b1-4bca6296d983\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") "
Jul 27 17:44:44.129648 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129419    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4be4456-1e6a-48d5-91b1-4bca6296d983-tls-certs\") pod \"c4be4456-1e6a-48d5-91b1-4bca6296d983\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") "
Jul 27 17:44:44.129648 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129478    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-model-cache\") pod \"c4be4456-1e6a-48d5-91b1-4bca6296d983\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") "
Jul 27 17:44:44.129648 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129537    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-dshm\") pod \"c4be4456-1e6a-48d5-91b1-4bca6296d983\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") "
Jul 27 17:44:44.129648 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129583    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-tmp-dir\") pod \"c4be4456-1e6a-48d5-91b1-4bca6296d983\" (UID: \"c4be4456-1e6a-48d5-91b1-4bca6296d983\") "
Jul 27 17:44:44.129858 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129722    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-home" (OuterVolumeSpecName: "home") pod "c4be4456-1e6a-48d5-91b1-4bca6296d983" (UID: "c4be4456-1e6a-48d5-91b1-4bca6296d983"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:44.129858 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129766    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4be4456-1e6a-48d5-91b1-4bca6296d983" (UID: "c4be4456-1e6a-48d5-91b1-4bca6296d983"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:44.129858 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129853    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:44.130001 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129897    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:44.130001 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.129924    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4be4456-1e6a-48d5-91b1-4bca6296d983" (UID: "c4be4456-1e6a-48d5-91b1-4bca6296d983"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:44.131693 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.131673    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-dshm" (OuterVolumeSpecName: "dshm") pod "c4be4456-1e6a-48d5-91b1-4bca6296d983" (UID: "c4be4456-1e6a-48d5-91b1-4bca6296d983"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:44.131777 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.131713    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4be4456-1e6a-48d5-91b1-4bca6296d983-kube-api-access-wxtgw" (OuterVolumeSpecName: "kube-api-access-wxtgw") pod "c4be4456-1e6a-48d5-91b1-4bca6296d983" (UID: "c4be4456-1e6a-48d5-91b1-4bca6296d983"). InnerVolumeSpecName "kube-api-access-wxtgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:44:44.131777 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.131742    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4be4456-1e6a-48d5-91b1-4bca6296d983-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4be4456-1e6a-48d5-91b1-4bca6296d983" (UID: "c4be4456-1e6a-48d5-91b1-4bca6296d983"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:44:44.230336 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.230282    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxtgw\" (UniqueName: \"kubernetes.io/projected/c4be4456-1e6a-48d5-91b1-4bca6296d983-kube-api-access-wxtgw\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:44.230336 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.230330    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4be4456-1e6a-48d5-91b1-4bca6296d983-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:44.230336 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.230339    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:44.230561 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.230348    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4be4456-1e6a-48d5-91b1-4bca6296d983-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:44.591144 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.591109    2565 generic.go:356] "Generic (PLEG): container finished" podID="c4be4456-1e6a-48d5-91b1-4bca6296d983" containerID="a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834" exitCode=0
Jul 27 17:44:44.591315 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.591147    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"
Jul 27 17:44:44.591315 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.591194    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx" event={"ID":"c4be4456-1e6a-48d5-91b1-4bca6296d983","Type":"ContainerDied","Data":"a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834"}
Jul 27 17:44:44.591315 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.591237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx" event={"ID":"c4be4456-1e6a-48d5-91b1-4bca6296d983","Type":"ContainerDied","Data":"98c9d8ee5750337a7c91431a0ea786a224da84ecdbd5c49586b6e8bad571bd93"}
Jul 27 17:44:44.591315 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.591255    2565 scope.go:117] "RemoveContainer" containerID="a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834"
Jul 27 17:44:44.600576 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.600560    2565 scope.go:117] "RemoveContainer" containerID="a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834"
Jul 27 17:44:44.600812 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:44:44.600794    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834\": container with ID starting with a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834 not found: ID does not exist" containerID="a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834"
Jul 27 17:44:44.600863 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.600820    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834"} err="failed to get container status \"a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834\": rpc error: code = NotFound desc = could not find container \"a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834\": container with ID starting with a58b57dbdde76ff3b74d48815310e7c06f1dbddc1bac8ddbe20b4b1f09a13834 not found: ID does not exist"
Jul 27 17:44:44.614063 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.614040    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"]
Jul 27 17:44:44.617282 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.617257    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-01b5eaab-kserve-646d4bf94c-wjczx"]
Jul 27 17:44:44.890474 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:44.890393    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7" podUID="80cf6585-02b1-41a7-b624-286b00d8f6d7" containerName="main" containerID="cri-o://67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf" gracePeriod=45
Jul 27 17:44:45.143214 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.143154    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:45.239180 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239146    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-home\") pod \"80cf6585-02b1-41a7-b624-286b00d8f6d7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") "
Jul 27 17:44:45.239379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239201    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-tmp-dir\") pod \"80cf6585-02b1-41a7-b624-286b00d8f6d7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") "
Jul 27 17:44:45.239379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239233    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-model-cache\") pod \"80cf6585-02b1-41a7-b624-286b00d8f6d7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") "
Jul 27 17:44:45.239379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239260    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80cf6585-02b1-41a7-b624-286b00d8f6d7-tls-certs\") pod \"80cf6585-02b1-41a7-b624-286b00d8f6d7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") "
Jul 27 17:44:45.239379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239313    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-dshm\") pod \"80cf6585-02b1-41a7-b624-286b00d8f6d7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") "
Jul 27 17:44:45.239379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239359    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfbkd\" (UniqueName: \"kubernetes.io/projected/80cf6585-02b1-41a7-b624-286b00d8f6d7-kube-api-access-rfbkd\") pod \"80cf6585-02b1-41a7-b624-286b00d8f6d7\" (UID: \"80cf6585-02b1-41a7-b624-286b00d8f6d7\") "
Jul 27 17:44:45.239637 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239438    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-home" (OuterVolumeSpecName: "home") pod "80cf6585-02b1-41a7-b624-286b00d8f6d7" (UID: "80cf6585-02b1-41a7-b624-286b00d8f6d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:45.239637 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239555    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80cf6585-02b1-41a7-b624-286b00d8f6d7" (UID: "80cf6585-02b1-41a7-b624-286b00d8f6d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:45.239637 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239567    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "80cf6585-02b1-41a7-b624-286b00d8f6d7" (UID: "80cf6585-02b1-41a7-b624-286b00d8f6d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:45.239764 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239668    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:45.239764 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239687    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:45.239764 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.239703    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:45.241459 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.241439    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-dshm" (OuterVolumeSpecName: "dshm") pod "80cf6585-02b1-41a7-b624-286b00d8f6d7" (UID: "80cf6585-02b1-41a7-b624-286b00d8f6d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:44:45.241518 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.241443    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80cf6585-02b1-41a7-b624-286b00d8f6d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80cf6585-02b1-41a7-b624-286b00d8f6d7" (UID: "80cf6585-02b1-41a7-b624-286b00d8f6d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:44:45.241549 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.241518    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80cf6585-02b1-41a7-b624-286b00d8f6d7-kube-api-access-rfbkd" (OuterVolumeSpecName: "kube-api-access-rfbkd") pod "80cf6585-02b1-41a7-b624-286b00d8f6d7" (UID: "80cf6585-02b1-41a7-b624-286b00d8f6d7"). InnerVolumeSpecName "kube-api-access-rfbkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:44:45.255642 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.255617    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4be4456-1e6a-48d5-91b1-4bca6296d983" path="/var/lib/kubelet/pods/c4be4456-1e6a-48d5-91b1-4bca6296d983/volumes"
Jul 27 17:44:45.340453 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.340427    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80cf6585-02b1-41a7-b624-286b00d8f6d7-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:45.340453 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.340449    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80cf6585-02b1-41a7-b624-286b00d8f6d7-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:45.340453 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.340459    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfbkd\" (UniqueName: \"kubernetes.io/projected/80cf6585-02b1-41a7-b624-286b00d8f6d7-kube-api-access-rfbkd\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:44:45.564694 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.564652    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:44:45.596880 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.596849    2565 generic.go:356] "Generic (PLEG): container finished" podID="80cf6585-02b1-41a7-b624-286b00d8f6d7" containerID="67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf" exitCode=0
Jul 27 17:44:45.597025 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.596890    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"
Jul 27 17:44:45.597025 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.596930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7" event={"ID":"80cf6585-02b1-41a7-b624-286b00d8f6d7","Type":"ContainerDied","Data":"67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf"}
Jul 27 17:44:45.597025 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.596963    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7" event={"ID":"80cf6585-02b1-41a7-b624-286b00d8f6d7","Type":"ContainerDied","Data":"8779eec552bca7b87cdb15348901eb988d79fed4a102c5f2258bbb250e08a3d5"}
Jul 27 17:44:45.597025 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.596981    2565 scope.go:117] "RemoveContainer" containerID="67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf"
Jul 27 17:44:45.605255 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.605241    2565 scope.go:117] "RemoveContainer" containerID="67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf"
Jul 27 17:44:45.605503 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:44:45.605487    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf\": container with ID starting with 67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf not found: ID does not exist" containerID="67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf"
Jul 27 17:44:45.605575 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.605509    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf"} err="failed to get container status \"67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf\": rpc error: code = NotFound desc = could not find container \"67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf\": container with ID starting with 67b46467540393419bfbcef1ebbddb69f1fdecbe5762e095cb02c97940d33ddf not found: ID does not exist"
Jul 27 17:44:45.612901 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.612880    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"]
Jul 27 17:44:45.616665 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:45.616646    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-01b5eaab-kserve-66f86cfff-ssbh7"]
Jul 27 17:44:47.256624 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:47.256591    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80cf6585-02b1-41a7-b624-286b00d8f6d7" path="/var/lib/kubelet/pods/80cf6585-02b1-41a7-b624-286b00d8f6d7/volumes"
Jul 27 17:44:55.371123 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:44:55.371085    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:45:07.375696 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:07.375651    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:45:16.569873 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:16.569836    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:45:25.567880 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:25.567845    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:45:35.671988 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:35.671954    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:45:43.244054 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:43.244013    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:45:43.245992 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:43.245969    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:45:45.375840 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:45.375808    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:45:55.178647 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:45:55.178612    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:46:05.880035 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:46:05.879959    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:46:16.777903 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:46:16.777866    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:46:26.172416 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:46:26.172375    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:46:35.969926 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:46:35.969893    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:46:49.368153 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:46:49.368118    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:46:57.071548 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:46:57.071514    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:47:09.406761 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:47:09.406728    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:47:22.269822 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:47:22.269779    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:47:33.461760 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:47:33.461724    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:47:44.550014 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:47:44.549928    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:48:00.378549 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:00.378517    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:48:13.565064 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:13.565030    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:48:33.953991 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.953957    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"]
Jul 27 17:48:33.954466 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.954281    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4be4456-1e6a-48d5-91b1-4bca6296d983" containerName="main"
Jul 27 17:48:33.954466 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.954306    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4be4456-1e6a-48d5-91b1-4bca6296d983" containerName="main"
Jul 27 17:48:33.954466 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.954338    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80cf6585-02b1-41a7-b624-286b00d8f6d7" containerName="main"
Jul 27 17:48:33.954466 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.954344    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="80cf6585-02b1-41a7-b624-286b00d8f6d7" containerName="main"
Jul 27 17:48:33.954466 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.954409    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="80cf6585-02b1-41a7-b624-286b00d8f6d7" containerName="main"
Jul 27 17:48:33.954466 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.954421    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4be4456-1e6a-48d5-91b1-4bca6296d983" containerName="main"
Jul 27 17:48:33.957330 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.957311    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"
Jul 27 17:48:33.960156 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.960133    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 17:48:33.960279 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.960172    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 17:48:33.965028 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:33.965010    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"]
Jul 27 17:48:34.047584 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:34.047554    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4996e317-eaf5-4442-b6e7-2006180e4880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5\" (UID: \"4996e317-eaf5-4442-b6e7-2006180e4880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"
Jul 27 17:48:34.148334 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:34.148285    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4996e317-eaf5-4442-b6e7-2006180e4880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5\" (UID: \"4996e317-eaf5-4442-b6e7-2006180e4880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"
Jul 27 17:48:34.148938 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:34.148913    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4996e317-eaf5-4442-b6e7-2006180e4880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5\" (UID: \"4996e317-eaf5-4442-b6e7-2006180e4880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"
Jul 27 17:48:34.268403 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:34.268335    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"
Jul 27 17:48:34.395041 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:34.395018    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"]
Jul 27 17:48:34.397095 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:48:34.397068    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4996e317_eaf5_4442_b6e7_2006180e4880.slice/crio-b5dc3121d24b0e2617acb06305790fcaab4798c78c43f6ecfcba16162069492f WatchSource:0}: Error finding container b5dc3121d24b0e2617acb06305790fcaab4798c78c43f6ecfcba16162069492f: Status 404 returned error can't find the container with id b5dc3121d24b0e2617acb06305790fcaab4798c78c43f6ecfcba16162069492f
Jul 27 17:48:34.398869 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:34.398854    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:48:34.612941 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:34.612909    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5" event={"ID":"4996e317-eaf5-4442-b6e7-2006180e4880","Type":"ContainerStarted","Data":"b5dc3121d24b0e2617acb06305790fcaab4798c78c43f6ecfcba16162069492f"}
Jul 27 17:48:37.628548 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:37.628509    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5" event={"ID":"4996e317-eaf5-4442-b6e7-2006180e4880","Type":"ContainerStarted","Data":"47fa6f64a5005de1daf17d4c168485715a4b33a806759a65bcf159b1b8769c93"}
Jul 27 17:48:37.644903 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:37.644857    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5" podStartSLOduration=1.842049962 podStartE2EDuration="4.644844264s" totalImagesPullingTime="2.802794302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:48:37.64451455 +0000 UTC m=+1674.905489750" watchObservedRunningTime="2026-07-27 17:48:37.644844264 +0000 UTC m=+1674.905819463"
Jul 27 17:48:38.634012 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:38.633974    2565 generic.go:356] "Generic (PLEG): container finished" podID="4996e317-eaf5-4442-b6e7-2006180e4880" containerID="47fa6f64a5005de1daf17d4c168485715a4b33a806759a65bcf159b1b8769c93" exitCode=0
Jul 27 17:48:38.634406 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:38.634065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5" event={"ID":"4996e317-eaf5-4442-b6e7-2006180e4880","Type":"ContainerDied","Data":"47fa6f64a5005de1daf17d4c168485715a4b33a806759a65bcf159b1b8769c93"}
Jul 27 17:48:39.764338 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:39.764315    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"
Jul 27 17:48:39.899163 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:39.899084    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4996e317-eaf5-4442-b6e7-2006180e4880-maas-api-service-ca\") pod \"4996e317-eaf5-4442-b6e7-2006180e4880\" (UID: \"4996e317-eaf5-4442-b6e7-2006180e4880\") "
Jul 27 17:48:39.899438 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:39.899415    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4996e317-eaf5-4442-b6e7-2006180e4880-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4996e317-eaf5-4442-b6e7-2006180e4880" (UID: "4996e317-eaf5-4442-b6e7-2006180e4880"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:48:40.000611 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:40.000579    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4996e317-eaf5-4442-b6e7-2006180e4880-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:48:40.644131 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:40.644100    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"
Jul 27 17:48:40.644323 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:40.644144    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5" event={"ID":"4996e317-eaf5-4442-b6e7-2006180e4880","Type":"ContainerDied","Data":"b5dc3121d24b0e2617acb06305790fcaab4798c78c43f6ecfcba16162069492f"}
Jul 27 17:48:40.644323 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:40.644190    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5dc3121d24b0e2617acb06305790fcaab4798c78c43f6ecfcba16162069492f"
Jul 27 17:48:48.467048 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:48.467013    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:48:49.029781 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:49.029748    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"]
Jul 27 17:48:49.032403 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:49.032378    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5bebd9b0-4zzp5"]
Jul 27 17:48:49.255872 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:48:49.255840    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4996e317-eaf5-4442-b6e7-2006180e4880" path="/var/lib/kubelet/pods/4996e317-eaf5-4442-b6e7-2006180e4880/volumes"
Jul 27 17:49:01.771113 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:49:01.771079    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:49:30.868772 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:49:30.868741    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:49:42.683245 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:49:42.683208    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:50:01.640815 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.640781    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"]
Jul 27 17:50:01.641230 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.641126    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4996e317-eaf5-4442-b6e7-2006180e4880" containerName="revoke-keys"
Jul 27 17:50:01.641230 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.641139    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4996e317-eaf5-4442-b6e7-2006180e4880" containerName="revoke-keys"
Jul 27 17:50:01.641230 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.641227    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4996e317-eaf5-4442-b6e7-2006180e4880" containerName="revoke-keys"
Jul 27 17:50:01.644202 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.644182    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"
Jul 27 17:50:01.647048 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.647030    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 17:50:01.647140 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.647049    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 17:50:01.652206 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.652181    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"]
Jul 27 17:50:01.699046 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.699017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ccc7cf8-dc03-439c-b574-ad96647da938-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v\" (UID: \"7ccc7cf8-dc03-439c-b574-ad96647da938\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"
Jul 27 17:50:01.800169 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.800130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ccc7cf8-dc03-439c-b574-ad96647da938-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v\" (UID: \"7ccc7cf8-dc03-439c-b574-ad96647da938\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"
Jul 27 17:50:01.800751 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.800729    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ccc7cf8-dc03-439c-b574-ad96647da938-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v\" (UID: \"7ccc7cf8-dc03-439c-b574-ad96647da938\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"
Jul 27 17:50:01.955115 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:01.955026    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"
Jul 27 17:50:02.087326 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:02.087285    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"]
Jul 27 17:50:02.089373 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:50:02.089338    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ccc7cf8_dc03_439c_b574_ad96647da938.slice/crio-378d836f52a09e3806560b2d19677d3856969a5b3515e43d49aa977ac9f28855 WatchSource:0}: Error finding container 378d836f52a09e3806560b2d19677d3856969a5b3515e43d49aa977ac9f28855: Status 404 returned error can't find the container with id 378d836f52a09e3806560b2d19677d3856969a5b3515e43d49aa977ac9f28855
Jul 27 17:50:03.001951 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:03.001907    2565 generic.go:356] "Generic (PLEG): container finished" podID="7ccc7cf8-dc03-439c-b574-ad96647da938" containerID="c8e86e9e4f869be3651251d9b7023b832cb04252fb5b26cf7fd509e92ab8f9d7" exitCode=0
Jul 27 17:50:03.002414 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:03.001998    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v" event={"ID":"7ccc7cf8-dc03-439c-b574-ad96647da938","Type":"ContainerDied","Data":"c8e86e9e4f869be3651251d9b7023b832cb04252fb5b26cf7fd509e92ab8f9d7"}
Jul 27 17:50:03.002414 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:03.002074    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v" event={"ID":"7ccc7cf8-dc03-439c-b574-ad96647da938","Type":"ContainerStarted","Data":"378d836f52a09e3806560b2d19677d3856969a5b3515e43d49aa977ac9f28855"}
Jul 27 17:50:04.135110 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:04.135080    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"
Jul 27 17:50:04.220264 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:04.220214    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ccc7cf8-dc03-439c-b574-ad96647da938-maas-api-service-ca\") pod \"7ccc7cf8-dc03-439c-b574-ad96647da938\" (UID: \"7ccc7cf8-dc03-439c-b574-ad96647da938\") "
Jul 27 17:50:04.220616 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:04.220593    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ccc7cf8-dc03-439c-b574-ad96647da938-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ccc7cf8-dc03-439c-b574-ad96647da938" (UID: "7ccc7cf8-dc03-439c-b574-ad96647da938"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:50:04.321120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:04.321080    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ccc7cf8-dc03-439c-b574-ad96647da938-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:50:05.011854 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:05.011827    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"
Jul 27 17:50:05.011854 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:05.011836    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v" event={"ID":"7ccc7cf8-dc03-439c-b574-ad96647da938","Type":"ContainerDied","Data":"378d836f52a09e3806560b2d19677d3856969a5b3515e43d49aa977ac9f28855"}
Jul 27 17:50:05.012067 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:05.011889    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="378d836f52a09e3806560b2d19677d3856969a5b3515e43d49aa977ac9f28855"
Jul 27 17:50:15.066736 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:15.066703    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:50:16.729304 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:16.729261    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"]
Jul 27 17:50:16.730112 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:16.730090    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-837a3b5f-xtg2v"]
Jul 27 17:50:17.256019 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:17.255974    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ccc7cf8-dc03-439c-b574-ad96647da938" path="/var/lib/kubelet/pods/7ccc7cf8-dc03-439c-b574-ad96647da938/volumes"
Jul 27 17:50:26.669508 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:26.669468    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:50:43.280498 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:43.280468    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:50:43.283216 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:43.283193    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:50:59.386858 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:50:59.386816    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:52:26.368551 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:26.368512    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:52:36.473767 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:36.473736    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:52:46.565067 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:46.565035    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:52:54.364855 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.364819    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"]
Jul 27 17:52:54.365380 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.365339    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ccc7cf8-dc03-439c-b574-ad96647da938" containerName="revoke-keys"
Jul 27 17:52:54.365380 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.365359    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ccc7cf8-dc03-439c-b574-ad96647da938" containerName="revoke-keys"
Jul 27 17:52:54.365510 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.365494    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ccc7cf8-dc03-439c-b574-ad96647da938" containerName="revoke-keys"
Jul 27 17:52:54.368448 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.368428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"
Jul 27 17:52:54.371207 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.371184    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 17:52:54.371207 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.371197    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 17:52:54.375259 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.375238    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"]
Jul 27 17:52:54.482531 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.482493    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2857d6f-620a-4142-8e4e-988c72049a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3852632a-qb9wf\" (UID: \"f2857d6f-620a-4142-8e4e-988c72049a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"
Jul 27 17:52:54.583921 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.583872    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2857d6f-620a-4142-8e4e-988c72049a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3852632a-qb9wf\" (UID: \"f2857d6f-620a-4142-8e4e-988c72049a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"
Jul 27 17:52:54.584584 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.584561    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2857d6f-620a-4142-8e4e-988c72049a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3852632a-qb9wf\" (UID: \"f2857d6f-620a-4142-8e4e-988c72049a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"
Jul 27 17:52:54.679588 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.679514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"
Jul 27 17:52:54.811667 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:54.811646    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"]
Jul 27 17:52:54.813899 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:52:54.813869    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2857d6f_620a_4142_8e4e_988c72049a1a.slice/crio-e2697eb0cc1ec216b08a1baec13e854c0c84b6a3a6237ccbabf64b207a34379a WatchSource:0}: Error finding container e2697eb0cc1ec216b08a1baec13e854c0c84b6a3a6237ccbabf64b207a34379a: Status 404 returned error can't find the container with id e2697eb0cc1ec216b08a1baec13e854c0c84b6a3a6237ccbabf64b207a34379a
Jul 27 17:52:55.796333 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:55.796233    2565 generic.go:356] "Generic (PLEG): container finished" podID="f2857d6f-620a-4142-8e4e-988c72049a1a" containerID="d5a6639274e772d32b5e237742049f88e931ab8bf2876f7aa7c1f6553d51b710" exitCode=0
Jul 27 17:52:55.796333 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:55.796284    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf" event={"ID":"f2857d6f-620a-4142-8e4e-988c72049a1a","Type":"ContainerDied","Data":"d5a6639274e772d32b5e237742049f88e931ab8bf2876f7aa7c1f6553d51b710"}
Jul 27 17:52:55.796714 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:55.796376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf" event={"ID":"f2857d6f-620a-4142-8e4e-988c72049a1a","Type":"ContainerStarted","Data":"e2697eb0cc1ec216b08a1baec13e854c0c84b6a3a6237ccbabf64b207a34379a"}
Jul 27 17:52:56.927433 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:56.927407    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"
Jul 27 17:52:57.002884 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:57.002852    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2857d6f-620a-4142-8e4e-988c72049a1a-maas-api-service-ca\") pod \"f2857d6f-620a-4142-8e4e-988c72049a1a\" (UID: \"f2857d6f-620a-4142-8e4e-988c72049a1a\") "
Jul 27 17:52:57.003198 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:57.003176    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2857d6f-620a-4142-8e4e-988c72049a1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f2857d6f-620a-4142-8e4e-988c72049a1a" (UID: "f2857d6f-620a-4142-8e4e-988c72049a1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:52:57.103854 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:57.103771    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2857d6f-620a-4142-8e4e-988c72049a1a-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:52:57.807493 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:57.807458    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf" event={"ID":"f2857d6f-620a-4142-8e4e-988c72049a1a","Type":"ContainerDied","Data":"e2697eb0cc1ec216b08a1baec13e854c0c84b6a3a6237ccbabf64b207a34379a"}
Jul 27 17:52:57.807698 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:57.807508    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2697eb0cc1ec216b08a1baec13e854c0c84b6a3a6237ccbabf64b207a34379a"
Jul 27 17:52:57.807698 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:52:57.807470    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"
Jul 27 17:53:07.775964 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:53:07.775929    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:53:09.478737 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:53:09.478695    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"]
Jul 27 17:53:09.481422 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:53:09.481395    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3852632a-qb9wf"]
Jul 27 17:53:11.255721 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:53:11.255687    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2857d6f-620a-4142-8e4e-988c72049a1a" path="/var/lib/kubelet/pods/f2857d6f-620a-4142-8e4e-988c72049a1a/volumes"
Jul 27 17:53:44.169752 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:53:44.169669    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:54:09.365960 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.365927    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"]
Jul 27 17:54:09.366472 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.366455    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2857d6f-620a-4142-8e4e-988c72049a1a" containerName="revoke-keys"
Jul 27 17:54:09.366521 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.366476    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2857d6f-620a-4142-8e4e-988c72049a1a" containerName="revoke-keys"
Jul 27 17:54:09.366585 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.366574    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2857d6f-620a-4142-8e4e-988c72049a1a" containerName="revoke-keys"
Jul 27 17:54:09.369876 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.369856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.372774 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.372754    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1b693e0c\"/\"kube-root-ca.crt\""
Jul 27 17:54:09.374012 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.373989    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1b693e0c\"/\"default-dockercfg-vlb5c\""
Jul 27 17:54:09.374134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.373995    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1b693e0c\"/\"openshift-service-ca.crt\""
Jul 27 17:54:09.374477 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.374455    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1b693e0c\"/\"e2e-lifecycle-model-1b693e0c-kserve-self-signed-certs\""
Jul 27 17:54:09.377022 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.377001    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"]
Jul 27 17:54:09.501553 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.501516    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-dshm\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.501553 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.501556    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-tmp-dir\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.501787 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.501575    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c61457-dc9d-4479-8d16-073230dfa133-tls-certs\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.501787 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.501681    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-home\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.501787 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.501710    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-model-cache\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.501787 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.501749    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr4vd\" (UniqueName: \"kubernetes.io/projected/d4c61457-dc9d-4479-8d16-073230dfa133-kube-api-access-mr4vd\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.602775 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.602741    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr4vd\" (UniqueName: \"kubernetes.io/projected/d4c61457-dc9d-4479-8d16-073230dfa133-kube-api-access-mr4vd\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.602969 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.602788    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-dshm\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.602969 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.602810    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-tmp-dir\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.602969 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.602829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c61457-dc9d-4479-8d16-073230dfa133-tls-certs\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.602969 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.602873    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-home\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.602969 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.602886    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-model-cache\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.603195 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.603156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-tmp-dir\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.603284 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.603262    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-model-cache\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.603356 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.603344    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-home\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.605120 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.605093    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-dshm\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.605239 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.605223    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c61457-dc9d-4479-8d16-073230dfa133-tls-certs\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.611563 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.611538    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr4vd\" (UniqueName: \"kubernetes.io/projected/d4c61457-dc9d-4479-8d16-073230dfa133-kube-api-access-mr4vd\") pod \"e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") " pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.680597 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.680546    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:09.807379 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:09.807352    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"]
Jul 27 17:54:09.809470 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:54:09.809443    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4c61457_dc9d_4479_8d16_073230dfa133.slice/crio-d870a5e9618cbb4725c999952513bd5ca08dd3703d682938727811d690f44593 WatchSource:0}: Error finding container d870a5e9618cbb4725c999952513bd5ca08dd3703d682938727811d690f44593: Status 404 returned error can't find the container with id d870a5e9618cbb4725c999952513bd5ca08dd3703d682938727811d690f44593
Jul 27 17:54:10.139779 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:10.139743    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh" event={"ID":"d4c61457-dc9d-4479-8d16-073230dfa133","Type":"ContainerStarted","Data":"ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146"}
Jul 27 17:54:10.139779 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:10.139781    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh" event={"ID":"d4c61457-dc9d-4479-8d16-073230dfa133","Type":"ContainerStarted","Data":"d870a5e9618cbb4725c999952513bd5ca08dd3703d682938727811d690f44593"}
Jul 27 17:54:10.159065 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:10.159017    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh" podStartSLOduration=1.1589995690000001 podStartE2EDuration="1.158999569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:54:10.156809534 +0000 UTC m=+2007.417784733" watchObservedRunningTime="2026-07-27 17:54:10.158999569 +0000 UTC m=+2007.419974768"
Jul 27 17:54:12.875887 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:12.875851    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:54:19.681126 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:19.681094    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:19.681555 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:19.681139    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:19.693731 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:19.693709    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:20.197092 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:20.197061    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:54:43.351339 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:43.351283    2565 scope.go:117] "RemoveContainer" containerID="47fa6f64a5005de1daf17d4c168485715a4b33a806759a65bcf159b1b8769c93"
Jul 27 17:54:43.886773 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:43.886739    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:54:48.586648 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.586617    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"]
Jul 27 17:54:48.589852 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.589835    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"
Jul 27 17:54:48.592742 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.592722    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 17:54:48.592856 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.592728    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 17:54:48.598663 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.598637    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"]
Jul 27 17:54:48.716973 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.716946    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49410ba5-fb65-4bce-878a-fd02953c6090-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm\" (UID: \"49410ba5-fb65-4bce-878a-fd02953c6090\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"
Jul 27 17:54:48.818289 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.818257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49410ba5-fb65-4bce-878a-fd02953c6090-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm\" (UID: \"49410ba5-fb65-4bce-878a-fd02953c6090\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"
Jul 27 17:54:48.818893 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.818875    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49410ba5-fb65-4bce-878a-fd02953c6090-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm\" (UID: \"49410ba5-fb65-4bce-878a-fd02953c6090\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"
Jul 27 17:54:48.901836 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:48.901779    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"
Jul 27 17:54:49.029019 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:49.028989    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"]
Jul 27 17:54:49.030964 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:54:49.030932    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49410ba5_fb65_4bce_878a_fd02953c6090.slice/crio-a0b6b2ed86f219f066e1ba568c3821d1e2b12c8bfd45951da16976dca7858a66 WatchSource:0}: Error finding container a0b6b2ed86f219f066e1ba568c3821d1e2b12c8bfd45951da16976dca7858a66: Status 404 returned error can't find the container with id a0b6b2ed86f219f066e1ba568c3821d1e2b12c8bfd45951da16976dca7858a66
Jul 27 17:54:49.321002 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:49.320974    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm" event={"ID":"49410ba5-fb65-4bce-878a-fd02953c6090","Type":"ContainerStarted","Data":"a0b6b2ed86f219f066e1ba568c3821d1e2b12c8bfd45951da16976dca7858a66"}
Jul 27 17:54:50.327223 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:50.327187    2565 generic.go:356] "Generic (PLEG): container finished" podID="49410ba5-fb65-4bce-878a-fd02953c6090" containerID="b90fb0dda58e5859c10747544c14140dadf3b020cf3cb4c480075bddbf3bbaf9" exitCode=0
Jul 27 17:54:50.327676 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:50.327233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm" event={"ID":"49410ba5-fb65-4bce-878a-fd02953c6090","Type":"ContainerDied","Data":"b90fb0dda58e5859c10747544c14140dadf3b020cf3cb4c480075bddbf3bbaf9"}
Jul 27 17:54:51.468070 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:51.468047    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"
Jul 27 17:54:51.545615 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:51.545586    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49410ba5-fb65-4bce-878a-fd02953c6090-maas-api-service-ca\") pod \"49410ba5-fb65-4bce-878a-fd02953c6090\" (UID: \"49410ba5-fb65-4bce-878a-fd02953c6090\") "
Jul 27 17:54:51.545915 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:51.545892    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49410ba5-fb65-4bce-878a-fd02953c6090-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49410ba5-fb65-4bce-878a-fd02953c6090" (UID: "49410ba5-fb65-4bce-878a-fd02953c6090"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:54:51.647103 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:51.647038    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49410ba5-fb65-4bce-878a-fd02953c6090-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:54:52.339153 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:52.339124    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"
Jul 27 17:54:52.339350 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:52.339163    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm" event={"ID":"49410ba5-fb65-4bce-878a-fd02953c6090","Type":"ContainerDied","Data":"a0b6b2ed86f219f066e1ba568c3821d1e2b12c8bfd45951da16976dca7858a66"}
Jul 27 17:54:52.339350 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:54:52.339211    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0b6b2ed86f219f066e1ba568c3821d1e2b12c8bfd45951da16976dca7858a66"
Jul 27 17:55:02.965604 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:02.965565    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:55:08.683327 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:08.683234    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"]
Jul 27 17:55:08.685383 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:08.685360    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b693e0c-f6hkm"]
Jul 27 17:55:09.255814 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:09.255777    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49410ba5-fb65-4bce-878a-fd02953c6090" path="/var/lib/kubelet/pods/49410ba5-fb65-4bce-878a-fd02953c6090/volumes"
Jul 27 17:55:11.663942 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:11.663908    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:55:16.859369 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:16.859334    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"]
Jul 27 17:55:31.878492 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:31.878431    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh" podUID="d4c61457-dc9d-4479-8d16-073230dfa133" containerName="main" containerID="cri-o://ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146" gracePeriod=45
Jul 27 17:55:32.132517 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.132454    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:55:32.201656 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.201626    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-home\") pod \"d4c61457-dc9d-4479-8d16-073230dfa133\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") "
Jul 27 17:55:32.201818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.201703    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-dshm\") pod \"d4c61457-dc9d-4479-8d16-073230dfa133\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") "
Jul 27 17:55:32.201818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.201721    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-model-cache\") pod \"d4c61457-dc9d-4479-8d16-073230dfa133\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") "
Jul 27 17:55:32.201818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.201741    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr4vd\" (UniqueName: \"kubernetes.io/projected/d4c61457-dc9d-4479-8d16-073230dfa133-kube-api-access-mr4vd\") pod \"d4c61457-dc9d-4479-8d16-073230dfa133\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") "
Jul 27 17:55:32.201818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.201775    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c61457-dc9d-4479-8d16-073230dfa133-tls-certs\") pod \"d4c61457-dc9d-4479-8d16-073230dfa133\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") "
Jul 27 17:55:32.201818 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.201808    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-tmp-dir\") pod \"d4c61457-dc9d-4479-8d16-073230dfa133\" (UID: \"d4c61457-dc9d-4479-8d16-073230dfa133\") "
Jul 27 17:55:32.202081 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.201935    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-home" (OuterVolumeSpecName: "home") pod "d4c61457-dc9d-4479-8d16-073230dfa133" (UID: "d4c61457-dc9d-4479-8d16-073230dfa133"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:55:32.202081 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.202041    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-model-cache" (OuterVolumeSpecName: "model-cache") pod "d4c61457-dc9d-4479-8d16-073230dfa133" (UID: "d4c61457-dc9d-4479-8d16-073230dfa133"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:55:32.202180 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.202061    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:55:32.202180 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.202111    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d4c61457-dc9d-4479-8d16-073230dfa133" (UID: "d4c61457-dc9d-4479-8d16-073230dfa133"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:55:32.204031 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.204002    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-dshm" (OuterVolumeSpecName: "dshm") pod "d4c61457-dc9d-4479-8d16-073230dfa133" (UID: "d4c61457-dc9d-4479-8d16-073230dfa133"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:55:32.204134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.204050    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c61457-dc9d-4479-8d16-073230dfa133-kube-api-access-mr4vd" (OuterVolumeSpecName: "kube-api-access-mr4vd") pod "d4c61457-dc9d-4479-8d16-073230dfa133" (UID: "d4c61457-dc9d-4479-8d16-073230dfa133"). InnerVolumeSpecName "kube-api-access-mr4vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:55:32.204134 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.204052    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4c61457-dc9d-4479-8d16-073230dfa133-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4c61457-dc9d-4479-8d16-073230dfa133" (UID: "d4c61457-dc9d-4479-8d16-073230dfa133"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:55:32.302799 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.302756    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:55:32.302799 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.302790    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:55:32.302799 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.302805    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr4vd\" (UniqueName: \"kubernetes.io/projected/d4c61457-dc9d-4479-8d16-073230dfa133-kube-api-access-mr4vd\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:55:32.303040 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.302816    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4c61457-dc9d-4479-8d16-073230dfa133-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:55:32.303040 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.302827    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4c61457-dc9d-4479-8d16-073230dfa133-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:55:32.524522 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.524430    2565 generic.go:356] "Generic (PLEG): container finished" podID="d4c61457-dc9d-4479-8d16-073230dfa133" containerID="ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146" exitCode=0
Jul 27 17:55:32.524522 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.524475    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"
Jul 27 17:55:32.524704 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.524512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh" event={"ID":"d4c61457-dc9d-4479-8d16-073230dfa133","Type":"ContainerDied","Data":"ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146"}
Jul 27 17:55:32.524704 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.524559    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh" event={"ID":"d4c61457-dc9d-4479-8d16-073230dfa133","Type":"ContainerDied","Data":"d870a5e9618cbb4725c999952513bd5ca08dd3703d682938727811d690f44593"}
Jul 27 17:55:32.524704 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.524576    2565 scope.go:117] "RemoveContainer" containerID="ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146"
Jul 27 17:55:32.533732 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.533709    2565 scope.go:117] "RemoveContainer" containerID="ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146"
Jul 27 17:55:32.533992 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:55:32.533972    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146\": container with ID starting with ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146 not found: ID does not exist" containerID="ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146"
Jul 27 17:55:32.534049 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.534000    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146"} err="failed to get container status \"ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146\": rpc error: code = NotFound desc = could not find container \"ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146\": container with ID starting with ecba8703299e580d440a469aa40c7f7a367faa36d8b342f0ac5a37e164ee6146 not found: ID does not exist"
Jul 27 17:55:32.545986 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.545963    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"]
Jul 27 17:55:32.550050 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:32.550031    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-1b693e0c/e2e-lifecycle-model-1b693e0c-kserve-686d9fbbc-wphxh"]
Jul 27 17:55:33.256039 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:33.256005    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c61457-dc9d-4479-8d16-073230dfa133" path="/var/lib/kubelet/pods/d4c61457-dc9d-4479-8d16-073230dfa133/volumes"
Jul 27 17:55:43.306944 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:43.306910    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:55:43.310790 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:55:43.310765    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 17:56:26.777273 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:56:26.777237    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:56:39.572600 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:56:39.572521    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:56:43.380458 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:56:43.380426    2565 scope.go:117] "RemoveContainer" containerID="c8e86e9e4f869be3651251d9b7023b832cb04252fb5b26cf7fd509e92ab8f9d7"
Jul 27 17:57:28.268593 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:28.268558    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:57:42.669556 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:42.669516    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:57:52.154411 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.154378    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"]
Jul 27 17:57:52.154784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.154680    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c61457-dc9d-4479-8d16-073230dfa133" containerName="main"
Jul 27 17:57:52.154784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.154690    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c61457-dc9d-4479-8d16-073230dfa133" containerName="main"
Jul 27 17:57:52.154784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.154702    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49410ba5-fb65-4bce-878a-fd02953c6090" containerName="revoke-keys"
Jul 27 17:57:52.154784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.154708    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="49410ba5-fb65-4bce-878a-fd02953c6090" containerName="revoke-keys"
Jul 27 17:57:52.154784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.154776    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c61457-dc9d-4479-8d16-073230dfa133" containerName="main"
Jul 27 17:57:52.154784 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.154786    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="49410ba5-fb65-4bce-878a-fd02953c6090" containerName="revoke-keys"
Jul 27 17:57:52.157802 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.157781    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.160313 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.160279    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a155ed\"/\"openshift-service-ca.crt\""
Jul 27 17:57:52.160463 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.160445    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a155ed\"/\"kube-root-ca.crt\""
Jul 27 17:57:52.161533 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.161513    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a155ed\"/\"default-dockercfg-gg6qk\""
Jul 27 17:57:52.161641 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.161626    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a155ed\"/\"test-model-a155ed-kserve-self-signed-certs\""
Jul 27 17:57:52.167108 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.167086    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"]
Jul 27 17:57:52.247370 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.247329    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tmp-dir\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.247539 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.247420    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-model-cache\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.247539 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.247472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-dshm\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.247539 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.247490    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tls-certs\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.247539 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.247528    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-home\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.247725 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.247577    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhgrd\" (UniqueName: \"kubernetes.io/projected/8a0204b1-12bc-4b6c-8e43-e616706d4f57-kube-api-access-xhgrd\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.347945 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.347911    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhgrd\" (UniqueName: \"kubernetes.io/projected/8a0204b1-12bc-4b6c-8e43-e616706d4f57-kube-api-access-xhgrd\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348146 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.347954    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tmp-dir\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348146 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.348013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-model-cache\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348146 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.348115    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-dshm\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348146 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.348137    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tls-certs\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348404 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.348191    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-home\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348586 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.348559    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-model-cache\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348718 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.348628    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tmp-dir\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.348718 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.348660    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-home\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.350532 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.350516    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-dshm\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.350688 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.350671    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tls-certs\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.356391 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.356370    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhgrd\" (UniqueName: \"kubernetes.io/projected/8a0204b1-12bc-4b6c-8e43-e616706d4f57-kube-api-access-xhgrd\") pod \"test-model-a155ed-kserve-76c68c87-jpcv2\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") " pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.467858 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.467787    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:57:52.592142 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:52.592119    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"]
Jul 27 17:57:52.593916 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:57:52.593890    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a0204b1_12bc_4b6c_8e43_e616706d4f57.slice/crio-217ef48e71668b0238d0194067f9e5763ceabc1c88d329d89de7058d5c44dfef WatchSource:0}: Error finding container 217ef48e71668b0238d0194067f9e5763ceabc1c88d329d89de7058d5c44dfef: Status 404 returned error can't find the container with id 217ef48e71668b0238d0194067f9e5763ceabc1c88d329d89de7058d5c44dfef
Jul 27 17:57:53.167139 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:53.167103    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2" event={"ID":"8a0204b1-12bc-4b6c-8e43-e616706d4f57","Type":"ContainerStarted","Data":"84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40"}
Jul 27 17:57:53.167139 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:53.167148    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2" event={"ID":"8a0204b1-12bc-4b6c-8e43-e616706d4f57","Type":"ContainerStarted","Data":"217ef48e71668b0238d0194067f9e5763ceabc1c88d329d89de7058d5c44dfef"}
Jul 27 17:57:53.184574 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:53.184530    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2" podStartSLOduration=1.184515316 podStartE2EDuration="1.184515316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:57:53.183320848 +0000 UTC m=+2230.444296046" watchObservedRunningTime="2026-07-27 17:57:53.184515316 +0000 UTC m=+2230.445490513"
Jul 27 17:57:55.759040 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:57:55.758988    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:58:02.468829 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:02.468797    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:58:02.469445 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:02.469094    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:58:02.481126 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:02.481106    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:58:03.222456 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:03.222428    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:58:07.260095 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:07.260023    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:58:14.261763 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.261726    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"]
Jul 27 17:58:14.265242 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.265222    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.269162 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.269142    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ab2501\"/\"default-dockercfg-65t74\""
Jul 27 17:58:14.269282 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.269179    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ab2501\"/\"kube-root-ca.crt\""
Jul 27 17:58:14.269282 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.269183    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ab2501\"/\"test-model-ab2501-kserve-self-signed-certs\""
Jul 27 17:58:14.269444 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.269183    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ab2501\"/\"openshift-service-ca.crt\""
Jul 27 17:58:14.274936 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.274914    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"]
Jul 27 17:58:14.329723 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.329697    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-home\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.329850 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.329737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tls-certs\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.329850 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.329805    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-dshm\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.329850 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.329832    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tmp-dir\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.329978 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.329874    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-model-cache\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.329978 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.329899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nxhw\" (UniqueName: \"kubernetes.io/projected/63faa723-6ec7-40c6-b06f-8c19f9d6b117-kube-api-access-8nxhw\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.430512 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430481    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-home\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.430674 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430525    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tls-certs\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.430674 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-dshm\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.430674 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430570    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tmp-dir\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.430674 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430594    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-model-cache\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.430674 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430612    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxhw\" (UniqueName: \"kubernetes.io/projected/63faa723-6ec7-40c6-b06f-8c19f9d6b117-kube-api-access-8nxhw\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.430953 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430928    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-home\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.431036 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.430998    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tmp-dir\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.431088 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.431061    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-model-cache\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.432851 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.432823    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-dshm\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.433073 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.433058    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tls-certs\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.438852 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.438836    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nxhw\" (UniqueName: \"kubernetes.io/projected/63faa723-6ec7-40c6-b06f-8c19f9d6b117-kube-api-access-8nxhw\") pod \"test-model-ab2501-kserve-588dcddbdc-4g8b8\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") " pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.574933 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.574894    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:14.697499 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:14.697472    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"]
Jul 27 17:58:14.700074 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:58:14.700048    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63faa723_6ec7_40c6_b06f_8c19f9d6b117.slice/crio-f425ce38f4e984016c4634e9a9ae7e18481ae576a756e9a98b81062c088431ad WatchSource:0}: Error finding container f425ce38f4e984016c4634e9a9ae7e18481ae576a756e9a98b81062c088431ad: Status 404 returned error can't find the container with id f425ce38f4e984016c4634e9a9ae7e18481ae576a756e9a98b81062c088431ad
Jul 27 17:58:15.269335 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:15.269281    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8" event={"ID":"63faa723-6ec7-40c6-b06f-8c19f9d6b117","Type":"ContainerStarted","Data":"7e33bb62630b736e30ff21b7eb8cb36f402fd7ac07c6b193b0e32fad468d83b5"}
Jul 27 17:58:15.269715 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:15.269346    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8" event={"ID":"63faa723-6ec7-40c6-b06f-8c19f9d6b117","Type":"ContainerStarted","Data":"f425ce38f4e984016c4634e9a9ae7e18481ae576a756e9a98b81062c088431ad"}
Jul 27 17:58:15.287226 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:15.287174    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8" podStartSLOduration=1.287160256 podStartE2EDuration="1.287160256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:58:15.285640849 +0000 UTC m=+2252.546616047" watchObservedRunningTime="2026-07-27 17:58:15.287160256 +0000 UTC m=+2252.548135454"
Jul 27 17:58:17.864199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:17.864148    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:58:24.575684 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:24.575641    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:24.576199 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:24.575697    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:24.587780 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:24.587754    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:25.327257 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:25.327229    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:58:32.065748 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:32.065713    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:58:53.378173 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:53.378137    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"]
Jul 27 17:58:54.366336 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:54.366289    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"]
Jul 27 17:58:58.361769 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:58:58.361736    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:59:08.396833 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.396782    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2" podUID="8a0204b1-12bc-4b6c-8e43-e616706d4f57" containerName="main" containerID="cri-o://84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40" gracePeriod=45
Jul 27 17:59:08.639215 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.639189    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:59:08.777316 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777214    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhgrd\" (UniqueName: \"kubernetes.io/projected/8a0204b1-12bc-4b6c-8e43-e616706d4f57-kube-api-access-xhgrd\") pod \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") "
Jul 27 17:59:08.777477 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777327    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-model-cache\") pod \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") "
Jul 27 17:59:08.777477 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777367    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tmp-dir\") pod \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") "
Jul 27 17:59:08.777477 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777392    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-home\") pod \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") "
Jul 27 17:59:08.777477 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777426    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tls-certs\") pod \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") "
Jul 27 17:59:08.777477 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777452    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-dshm\") pod \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\" (UID: \"8a0204b1-12bc-4b6c-8e43-e616706d4f57\") "
Jul 27 17:59:08.777710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777578    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a0204b1-12bc-4b6c-8e43-e616706d4f57" (UID: "8a0204b1-12bc-4b6c-8e43-e616706d4f57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:08.777710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777610    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a0204b1-12bc-4b6c-8e43-e616706d4f57" (UID: "8a0204b1-12bc-4b6c-8e43-e616706d4f57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:08.777710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777687    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-home" (OuterVolumeSpecName: "home") pod "8a0204b1-12bc-4b6c-8e43-e616706d4f57" (UID: "8a0204b1-12bc-4b6c-8e43-e616706d4f57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:08.777820 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777722    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:08.777820 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.777754    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:08.779531 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.779501    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-dshm" (OuterVolumeSpecName: "dshm") pod "8a0204b1-12bc-4b6c-8e43-e616706d4f57" (UID: "8a0204b1-12bc-4b6c-8e43-e616706d4f57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:08.779645 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.779596    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a0204b1-12bc-4b6c-8e43-e616706d4f57" (UID: "8a0204b1-12bc-4b6c-8e43-e616706d4f57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:59:08.779703 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.779648    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0204b1-12bc-4b6c-8e43-e616706d4f57-kube-api-access-xhgrd" (OuterVolumeSpecName: "kube-api-access-xhgrd") pod "8a0204b1-12bc-4b6c-8e43-e616706d4f57" (UID: "8a0204b1-12bc-4b6c-8e43-e616706d4f57"). InnerVolumeSpecName "kube-api-access-xhgrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:08.878098 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.878070    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a0204b1-12bc-4b6c-8e43-e616706d4f57-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:08.878098 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.878092    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:08.878098 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.878101    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhgrd\" (UniqueName: \"kubernetes.io/projected/8a0204b1-12bc-4b6c-8e43-e616706d4f57-kube-api-access-xhgrd\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:08.878325 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:08.878113    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a0204b1-12bc-4b6c-8e43-e616706d4f57-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:09.387947 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.387892    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8" podUID="63faa723-6ec7-40c6-b06f-8c19f9d6b117" containerName="main" containerID="cri-o://7e33bb62630b736e30ff21b7eb8cb36f402fd7ac07c6b193b0e32fad468d83b5" gracePeriod=45
Jul 27 17:59:09.513885 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.513811    2565 generic.go:356] "Generic (PLEG): container finished" podID="8a0204b1-12bc-4b6c-8e43-e616706d4f57" containerID="84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40" exitCode=0
Jul 27 17:59:09.513885 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.513855    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"
Jul 27 17:59:09.514362 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.513892    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2" event={"ID":"8a0204b1-12bc-4b6c-8e43-e616706d4f57","Type":"ContainerDied","Data":"84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40"}
Jul 27 17:59:09.514362 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.513939    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2" event={"ID":"8a0204b1-12bc-4b6c-8e43-e616706d4f57","Type":"ContainerDied","Data":"217ef48e71668b0238d0194067f9e5763ceabc1c88d329d89de7058d5c44dfef"}
Jul 27 17:59:09.514362 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.513974    2565 scope.go:117] "RemoveContainer" containerID="84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40"
Jul 27 17:59:09.515748 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.515725    2565 generic.go:356] "Generic (PLEG): container finished" podID="63faa723-6ec7-40c6-b06f-8c19f9d6b117" containerID="7e33bb62630b736e30ff21b7eb8cb36f402fd7ac07c6b193b0e32fad468d83b5" exitCode=0
Jul 27 17:59:09.515866 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.515791    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8" event={"ID":"63faa723-6ec7-40c6-b06f-8c19f9d6b117","Type":"ContainerDied","Data":"7e33bb62630b736e30ff21b7eb8cb36f402fd7ac07c6b193b0e32fad468d83b5"}
Jul 27 17:59:09.522629 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.522612    2565 scope.go:117] "RemoveContainer" containerID="84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40"
Jul 27 17:59:09.522882 ip-10-0-131-8 kubenswrapper[2565]: E0727 17:59:09.522863    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40\": container with ID starting with 84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40 not found: ID does not exist" containerID="84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40"
Jul 27 17:59:09.522930 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.522890    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40"} err="failed to get container status \"84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40\": rpc error: code = NotFound desc = could not find container \"84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40\": container with ID starting with 84c842abf2489550e6e80b729f2e5968cd697e1e211902c77c664e73e66fec40 not found: ID does not exist"
Jul 27 17:59:09.531472 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.531447    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"]
Jul 27 17:59:09.535839 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.535815    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a155ed/test-model-a155ed-kserve-76c68c87-jpcv2"]
Jul 27 17:59:09.637044 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.637018    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:59:09.785745 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.785667    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-home\") pod \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") "
Jul 27 17:59:09.785745 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.785713    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tls-certs\") pod \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") "
Jul 27 17:59:09.785745 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.785734    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-dshm\") pod \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") "
Jul 27 17:59:09.785982 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.785775    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-model-cache\") pod \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") "
Jul 27 17:59:09.785982 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.785822    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nxhw\" (UniqueName: \"kubernetes.io/projected/63faa723-6ec7-40c6-b06f-8c19f9d6b117-kube-api-access-8nxhw\") pod \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") "
Jul 27 17:59:09.785982 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.785872    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tmp-dir\") pod \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\" (UID: \"63faa723-6ec7-40c6-b06f-8c19f9d6b117\") "
Jul 27 17:59:09.786142 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.785978    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-home" (OuterVolumeSpecName: "home") pod "63faa723-6ec7-40c6-b06f-8c19f9d6b117" (UID: "63faa723-6ec7-40c6-b06f-8c19f9d6b117"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:09.786142 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.786034    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-model-cache" (OuterVolumeSpecName: "model-cache") pod "63faa723-6ec7-40c6-b06f-8c19f9d6b117" (UID: "63faa723-6ec7-40c6-b06f-8c19f9d6b117"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:09.786245 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.786141    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:09.786245 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.786174    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:09.786245 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.786211    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63faa723-6ec7-40c6-b06f-8c19f9d6b117" (UID: "63faa723-6ec7-40c6-b06f-8c19f9d6b117"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:09.788001 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.787970    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-dshm" (OuterVolumeSpecName: "dshm") pod "63faa723-6ec7-40c6-b06f-8c19f9d6b117" (UID: "63faa723-6ec7-40c6-b06f-8c19f9d6b117"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:09.788123 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.788018    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63faa723-6ec7-40c6-b06f-8c19f9d6b117-kube-api-access-8nxhw" (OuterVolumeSpecName: "kube-api-access-8nxhw") pod "63faa723-6ec7-40c6-b06f-8c19f9d6b117" (UID: "63faa723-6ec7-40c6-b06f-8c19f9d6b117"). InnerVolumeSpecName "kube-api-access-8nxhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:09.788123 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.788042    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63faa723-6ec7-40c6-b06f-8c19f9d6b117" (UID: "63faa723-6ec7-40c6-b06f-8c19f9d6b117"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:59:09.886552 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.886521    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:09.886552 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.886545    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63faa723-6ec7-40c6-b06f-8c19f9d6b117-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:09.886552 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.886553    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63faa723-6ec7-40c6-b06f-8c19f9d6b117-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:09.886552 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:09.886561    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nxhw\" (UniqueName: \"kubernetes.io/projected/63faa723-6ec7-40c6-b06f-8c19f9d6b117-kube-api-access-8nxhw\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:10.521819 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:10.521791    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"
Jul 27 17:59:10.522284 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:10.521827    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8" event={"ID":"63faa723-6ec7-40c6-b06f-8c19f9d6b117","Type":"ContainerDied","Data":"f425ce38f4e984016c4634e9a9ae7e18481ae576a756e9a98b81062c088431ad"}
Jul 27 17:59:10.522284 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:10.521876    2565 scope.go:117] "RemoveContainer" containerID="7e33bb62630b736e30ff21b7eb8cb36f402fd7ac07c6b193b0e32fad468d83b5"
Jul 27 17:59:10.543092 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:10.543068    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"]
Jul 27 17:59:10.549023 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:10.549003    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ab2501/test-model-ab2501-kserve-588dcddbdc-4g8b8"]
Jul 27 17:59:11.256158 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:11.256126    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63faa723-6ec7-40c6-b06f-8c19f9d6b117" path="/var/lib/kubelet/pods/63faa723-6ec7-40c6-b06f-8c19f9d6b117/volumes"
Jul 27 17:59:11.256549 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:11.256535    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a0204b1-12bc-4b6c-8e43-e616706d4f57" path="/var/lib/kubelet/pods/8a0204b1-12bc-4b6c-8e43-e616706d4f57/volumes"
Jul 27 17:59:13.966528 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:13.966492    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:59:16.492202 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.492169    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"]
Jul 27 17:59:16.492586 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.492521    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63faa723-6ec7-40c6-b06f-8c19f9d6b117" containerName="main"
Jul 27 17:59:16.492586 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.492534    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="63faa723-6ec7-40c6-b06f-8c19f9d6b117" containerName="main"
Jul 27 17:59:16.492586 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.492545    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0204b1-12bc-4b6c-8e43-e616706d4f57" containerName="main"
Jul 27 17:59:16.492586 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.492550    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0204b1-12bc-4b6c-8e43-e616706d4f57" containerName="main"
Jul 27 17:59:16.492710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.492610    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a0204b1-12bc-4b6c-8e43-e616706d4f57" containerName="main"
Jul 27 17:59:16.492710 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.492618    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="63faa723-6ec7-40c6-b06f-8c19f9d6b117" containerName="main"
Jul 27 17:59:16.496923 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.496904    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"
Jul 27 17:59:16.499816 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.499794    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 17:59:16.499940 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.499797    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 17:59:16.503230 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.503210    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"]
Jul 27 17:59:16.642832 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.642798    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cb5adb-1474-49ba-9b4a-7207809bbe6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq\" (UID: \"36cb5adb-1474-49ba-9b4a-7207809bbe6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"
Jul 27 17:59:16.743708 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.743619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cb5adb-1474-49ba-9b4a-7207809bbe6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq\" (UID: \"36cb5adb-1474-49ba-9b4a-7207809bbe6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"
Jul 27 17:59:16.744242 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.744224    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cb5adb-1474-49ba-9b4a-7207809bbe6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq\" (UID: \"36cb5adb-1474-49ba-9b4a-7207809bbe6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"
Jul 27 17:59:16.808694 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.808664    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"
Jul 27 17:59:16.933815 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:16.933784    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"]
Jul 27 17:59:16.936510 ip-10-0-131-8 kubenswrapper[2565]: W0727 17:59:16.936480    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36cb5adb_1474_49ba_9b4a_7207809bbe6e.slice/crio-6f82cf173b5579c4dc556a6c1ed0c2b1bb881609d886b328dbcc41e07a3cff20 WatchSource:0}: Error finding container 6f82cf173b5579c4dc556a6c1ed0c2b1bb881609d886b328dbcc41e07a3cff20: Status 404 returned error can't find the container with id 6f82cf173b5579c4dc556a6c1ed0c2b1bb881609d886b328dbcc41e07a3cff20
Jul 27 17:59:17.561023 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:17.560988    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq" event={"ID":"36cb5adb-1474-49ba-9b4a-7207809bbe6e","Type":"ContainerStarted","Data":"fc2504a3c414c08d57532591d20940a4d4abd56c48d976db983dcde1dfaf0aab"}
Jul 27 17:59:17.561402 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:17.561036    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq" event={"ID":"36cb5adb-1474-49ba-9b4a-7207809bbe6e","Type":"ContainerStarted","Data":"6f82cf173b5579c4dc556a6c1ed0c2b1bb881609d886b328dbcc41e07a3cff20"}
Jul 27 17:59:17.575598 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:17.575551    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq" podStartSLOduration=1.5755362979999998 podStartE2EDuration="1.575536298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:59:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:59:17.57471779 +0000 UTC m=+2314.835692989" watchObservedRunningTime="2026-07-27 17:59:17.575536298 +0000 UTC m=+2314.836511496"
Jul 27 17:59:18.572585 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:18.572550    2565 generic.go:356] "Generic (PLEG): container finished" podID="36cb5adb-1474-49ba-9b4a-7207809bbe6e" containerID="fc2504a3c414c08d57532591d20940a4d4abd56c48d976db983dcde1dfaf0aab" exitCode=0
Jul 27 17:59:18.572949 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:18.572640    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq" event={"ID":"36cb5adb-1474-49ba-9b4a-7207809bbe6e","Type":"ContainerDied","Data":"fc2504a3c414c08d57532591d20940a4d4abd56c48d976db983dcde1dfaf0aab"}
Jul 27 17:59:19.705611 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:19.705585    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"
Jul 27 17:59:19.767082 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:19.767052    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cb5adb-1474-49ba-9b4a-7207809bbe6e-maas-api-service-ca\") pod \"36cb5adb-1474-49ba-9b4a-7207809bbe6e\" (UID: \"36cb5adb-1474-49ba-9b4a-7207809bbe6e\") "
Jul 27 17:59:19.767480 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:19.767456    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36cb5adb-1474-49ba-9b4a-7207809bbe6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "36cb5adb-1474-49ba-9b4a-7207809bbe6e" (UID: "36cb5adb-1474-49ba-9b4a-7207809bbe6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:59:19.867953 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:19.867874    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cb5adb-1474-49ba-9b4a-7207809bbe6e-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 17:59:20.582478 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:20.582446    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"
Jul 27 17:59:20.582677 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:20.582451    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq" event={"ID":"36cb5adb-1474-49ba-9b4a-7207809bbe6e","Type":"ContainerDied","Data":"6f82cf173b5579c4dc556a6c1ed0c2b1bb881609d886b328dbcc41e07a3cff20"}
Jul 27 17:59:20.582677 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:20.582558    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f82cf173b5579c4dc556a6c1ed0c2b1bb881609d886b328dbcc41e07a3cff20"
Jul 27 17:59:29.965765 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:29.965727    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 17:59:31.565986 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:31.565944    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"]
Jul 27 17:59:31.570206 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:31.570172    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ab2501-7g4xq"]
Jul 27 17:59:33.256782 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:33.256745    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36cb5adb-1474-49ba-9b4a-7207809bbe6e" path="/var/lib/kubelet/pods/36cb5adb-1474-49ba-9b4a-7207809bbe6e/volumes"
Jul 27 17:59:43.411347 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:43.411254    2565 scope.go:117] "RemoveContainer" containerID="d5a6639274e772d32b5e237742049f88e931ab8bf2876f7aa7c1f6553d51b710"
Jul 27 17:59:55.270977 ip-10-0-131-8 kubenswrapper[2565]: I0727 17:59:55.270941    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:00:00.139255 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.139225    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt"]
Jul 27 18:00:00.139642 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.139544    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36cb5adb-1474-49ba-9b4a-7207809bbe6e" containerName="revoke-keys"
Jul 27 18:00:00.139642 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.139555    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="36cb5adb-1474-49ba-9b4a-7207809bbe6e" containerName="revoke-keys"
Jul 27 18:00:00.139642 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.139626    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="36cb5adb-1474-49ba-9b4a-7207809bbe6e" containerName="revoke-keys"
Jul 27 18:00:00.142518 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.142502    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt"
Jul 27 18:00:00.151254 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.151231    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"]
Jul 27 18:00:00.151506 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.151489    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:00:00.151896 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.151876    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt"
Jul 27 18:00:00.154929 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.154914    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"
Jul 27 18:00:00.160861 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.160841    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt"]
Jul 27 18:00:00.166481 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.166458    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"]
Jul 27 18:00:00.181485 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.181468    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"
Jul 27 18:00:00.336262 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.336178    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"]
Jul 27 18:00:00.338538 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:00:00.338503    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02193d13_df6e_4587_b5b0_a072498dd449.slice/crio-113db80fe93965a3cea874f32d0c27116e8aeae4eb00589b0a499abe6f1d5fd9 WatchSource:0}: Error finding container 113db80fe93965a3cea874f32d0c27116e8aeae4eb00589b0a499abe6f1d5fd9: Status 404 returned error can't find the container with id 113db80fe93965a3cea874f32d0c27116e8aeae4eb00589b0a499abe6f1d5fd9
Jul 27 18:00:00.520404 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.520276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt"]
Jul 27 18:00:00.523538 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:00:00.523499    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13561fba_5345_49ee_8c60_8fc0e066ec29.slice/crio-7a7e0a83a8b1fb628471b8a31fed28e3535c88613682240f0d83ad8dad24d439 WatchSource:0}: Error finding container 7a7e0a83a8b1fb628471b8a31fed28e3535c88613682240f0d83ad8dad24d439: Status 404 returned error can't find the container with id 7a7e0a83a8b1fb628471b8a31fed28e3535c88613682240f0d83ad8dad24d439
Jul 27 18:00:00.765761 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.765730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq" event={"ID":"02193d13-df6e-4587-b5b0-a072498dd449","Type":"ContainerStarted","Data":"c77b1efb74bcd49b41201af2b657e59a693f4a5812b9238138ab8d19802a54cb"}
Jul 27 18:00:00.765953 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.765784    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq" event={"ID":"02193d13-df6e-4587-b5b0-a072498dd449","Type":"ContainerStarted","Data":"113db80fe93965a3cea874f32d0c27116e8aeae4eb00589b0a499abe6f1d5fd9"}
Jul 27 18:00:00.766881 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.766860    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt" event={"ID":"13561fba-5345-49ee-8c60-8fc0e066ec29","Type":"ContainerStarted","Data":"7a7e0a83a8b1fb628471b8a31fed28e3535c88613682240f0d83ad8dad24d439"}
Jul 27 18:00:00.795057 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:00.794985    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq" podStartSLOduration=0.794974196 podStartE2EDuration="794.974196ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:00:00.792722619 +0000 UTC m=+2358.053697817" watchObservedRunningTime="2026-07-27 18:00:00.794974196 +0000 UTC m=+2358.055949395"
Jul 27 18:00:01.772839 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:01.772752    2565 generic.go:356] "Generic (PLEG): container finished" podID="02193d13-df6e-4587-b5b0-a072498dd449" containerID="c77b1efb74bcd49b41201af2b657e59a693f4a5812b9238138ab8d19802a54cb" exitCode=7
Jul 27 18:00:01.773266 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:01.772842    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq" event={"ID":"02193d13-df6e-4587-b5b0-a072498dd449","Type":"ContainerDied","Data":"c77b1efb74bcd49b41201af2b657e59a693f4a5812b9238138ab8d19802a54cb"}
Jul 27 18:00:01.773266 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:01.772973    2565 scope.go:117] "RemoveContainer" containerID="c77b1efb74bcd49b41201af2b657e59a693f4a5812b9238138ab8d19802a54cb"
Jul 27 18:00:01.774503 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:01.774482    2565 generic.go:356] "Generic (PLEG): container finished" podID="13561fba-5345-49ee-8c60-8fc0e066ec29" containerID="15abddf45b2c660b8944f051f21b47a883cb1eee8b9e00a24b1d41fe70321a34" exitCode=0
Jul 27 18:00:01.774605 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:01.774533    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt" event={"ID":"13561fba-5345-49ee-8c60-8fc0e066ec29","Type":"ContainerDied","Data":"15abddf45b2c660b8944f051f21b47a883cb1eee8b9e00a24b1d41fe70321a34"}
Jul 27 18:00:02.781005 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:02.780960    2565 generic.go:356] "Generic (PLEG): container finished" podID="02193d13-df6e-4587-b5b0-a072498dd449" containerID="71e8e0f7dc06dd320a0b3f9019c0f36d038f45d1549e52f4963f7307cb543e25" exitCode=0
Jul 27 18:00:02.781448 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:02.781026    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq" event={"ID":"02193d13-df6e-4587-b5b0-a072498dd449","Type":"ContainerDied","Data":"71e8e0f7dc06dd320a0b3f9019c0f36d038f45d1549e52f4963f7307cb543e25"}
Jul 27 18:00:02.781448 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:02.781077    2565 scope.go:117] "RemoveContainer" containerID="c77b1efb74bcd49b41201af2b657e59a693f4a5812b9238138ab8d19802a54cb"
Jul 27 18:00:02.913487 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:02.913459    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt"
Jul 27 18:00:03.788184 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:03.788152    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt"
Jul 27 18:00:03.788621 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:03.788190    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752920-vblmt" event={"ID":"13561fba-5345-49ee-8c60-8fc0e066ec29","Type":"ContainerDied","Data":"7a7e0a83a8b1fb628471b8a31fed28e3535c88613682240f0d83ad8dad24d439"}
Jul 27 18:00:03.788621 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:03.788263    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a7e0a83a8b1fb628471b8a31fed28e3535c88613682240f0d83ad8dad24d439"
Jul 27 18:00:03.918627 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:03.918599    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"
Jul 27 18:00:04.794026 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:04.793995    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"
Jul 27 18:00:04.794026 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:04.794006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq" event={"ID":"02193d13-df6e-4587-b5b0-a072498dd449","Type":"ContainerDied","Data":"113db80fe93965a3cea874f32d0c27116e8aeae4eb00589b0a499abe6f1d5fd9"}
Jul 27 18:00:04.794488 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:04.794049    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="113db80fe93965a3cea874f32d0c27116e8aeae4eb00589b0a499abe6f1d5fd9"
Jul 27 18:00:09.460403 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:09.460367    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:00:12.411054 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411023    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"]
Jul 27 18:00:12.411540 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411519    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02193d13-df6e-4587-b5b0-a072498dd449" containerName="cleanup"
Jul 27 18:00:12.411617 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411543    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="02193d13-df6e-4587-b5b0-a072498dd449" containerName="cleanup"
Jul 27 18:00:12.411617 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411562    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02193d13-df6e-4587-b5b0-a072498dd449" containerName="cleanup"
Jul 27 18:00:12.411617 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411571    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="02193d13-df6e-4587-b5b0-a072498dd449" containerName="cleanup"
Jul 27 18:00:12.411617 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411591    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13561fba-5345-49ee-8c60-8fc0e066ec29" containerName="cleanup"
Jul 27 18:00:12.411617 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411600    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="13561fba-5345-49ee-8c60-8fc0e066ec29" containerName="cleanup"
Jul 27 18:00:12.411867 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411704    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="02193d13-df6e-4587-b5b0-a072498dd449" containerName="cleanup"
Jul 27 18:00:12.411867 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411721    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="02193d13-df6e-4587-b5b0-a072498dd449" containerName="cleanup"
Jul 27 18:00:12.411867 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.411732    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="13561fba-5345-49ee-8c60-8fc0e066ec29" containerName="cleanup"
Jul 27 18:00:12.414898 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.414878    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"
Jul 27 18:00:12.417464 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.417444    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:00:12.417615 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.417597    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 18:00:12.421423 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.421400    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"]
Jul 27 18:00:12.518183 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.518154    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr\" (UID: \"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"
Jul 27 18:00:12.619177 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.619149    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr\" (UID: \"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"
Jul 27 18:00:12.619761 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.619741    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr\" (UID: \"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"
Jul 27 18:00:12.726556 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.726490    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"
Jul 27 18:00:12.880308 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:12.880266    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"]
Jul 27 18:00:12.881739 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:00:12.881717    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc521f7e_28e3_4cd4_89f1_1f670d1b2bb1.slice/crio-d85edce270e3544877bae753243a4e102aee39588374efc58dc163491a486f90 WatchSource:0}: Error finding container d85edce270e3544877bae753243a4e102aee39588374efc58dc163491a486f90: Status 404 returned error can't find the container with id d85edce270e3544877bae753243a4e102aee39588374efc58dc163491a486f90
Jul 27 18:00:13.840870 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:13.840833    2565 generic.go:356] "Generic (PLEG): container finished" podID="fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1" containerID="a98a1f9c0e44fbca6a05ec96746aaecf00b7b3e27273a8970ee0af54df90e97b" exitCode=0
Jul 27 18:00:13.841218 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:13.840886    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr" event={"ID":"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1","Type":"ContainerDied","Data":"a98a1f9c0e44fbca6a05ec96746aaecf00b7b3e27273a8970ee0af54df90e97b"}
Jul 27 18:00:13.841218 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:13.840935    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr" event={"ID":"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1","Type":"ContainerStarted","Data":"d85edce270e3544877bae753243a4e102aee39588374efc58dc163491a486f90"}
Jul 27 18:00:14.971125 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:14.971099    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"
Jul 27 18:00:15.040634 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:15.040610    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1-maas-api-service-ca\") pod \"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1\" (UID: \"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1\") "
Jul 27 18:00:15.040919 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:15.040899    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1" (UID: "fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:15.141906 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:15.141842    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:00:15.851018 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:15.850991    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"
Jul 27 18:00:15.851195 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:15.850987    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr" event={"ID":"fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1","Type":"ContainerDied","Data":"d85edce270e3544877bae753243a4e102aee39588374efc58dc163491a486f90"}
Jul 27 18:00:15.851195 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:15.851114    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d85edce270e3544877bae753243a4e102aee39588374efc58dc163491a486f90"
Jul 27 18:00:29.018754 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:29.018721    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"]
Jul 27 18:00:29.024099 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:29.024070    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-617fa4-29752920-gh7rq"]
Jul 27 18:00:29.255702 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:29.255667    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02193d13-df6e-4587-b5b0-a072498dd449" path="/var/lib/kubelet/pods/02193d13-df6e-4587-b5b0-a072498dd449/volumes"
Jul 27 18:00:31.297423 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:31.297388    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:00:32.502194 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:32.502155    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"]
Jul 27 18:00:32.503684 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:32.503660    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-617fa4-fsmgr"]
Jul 27 18:00:33.257308 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:33.257257    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1" path="/var/lib/kubelet/pods/fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1/volumes"
Jul 27 18:00:39.427109 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.427073    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"]
Jul 27 18:00:39.427518 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.427428    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1" containerName="revoke-keys"
Jul 27 18:00:39.427518 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.427441    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1" containerName="revoke-keys"
Jul 27 18:00:39.427518 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.427496    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc521f7e-28e3-4cd4-89f1-1f670d1b2bb1" containerName="revoke-keys"
Jul 27 18:00:39.431459 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.431442    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"
Jul 27 18:00:39.434110 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.434083    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 18:00:39.434267 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.434124    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:00:39.437640 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.437617    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"]
Jul 27 18:00:39.543605 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.543555    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04899f67-ccdb-4d1a-b678-aef97a978cce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79\" (UID: \"04899f67-ccdb-4d1a-b678-aef97a978cce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"
Jul 27 18:00:39.643935 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.643897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04899f67-ccdb-4d1a-b678-aef97a978cce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79\" (UID: \"04899f67-ccdb-4d1a-b678-aef97a978cce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"
Jul 27 18:00:39.644578 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.644559    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04899f67-ccdb-4d1a-b678-aef97a978cce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79\" (UID: \"04899f67-ccdb-4d1a-b678-aef97a978cce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"
Jul 27 18:00:39.743188 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.743114    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"
Jul 27 18:00:39.873032 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.872973    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"]
Jul 27 18:00:39.875371 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:00:39.875344    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04899f67_ccdb_4d1a_b678_aef97a978cce.slice/crio-08d728c5138310f95fe93d35dedab53f0de0c935f383696ad6cd806b85b34f96 WatchSource:0}: Error finding container 08d728c5138310f95fe93d35dedab53f0de0c935f383696ad6cd806b85b34f96: Status 404 returned error can't find the container with id 08d728c5138310f95fe93d35dedab53f0de0c935f383696ad6cd806b85b34f96
Jul 27 18:00:39.960316 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:39.960270    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79" event={"ID":"04899f67-ccdb-4d1a-b678-aef97a978cce","Type":"ContainerStarted","Data":"08d728c5138310f95fe93d35dedab53f0de0c935f383696ad6cd806b85b34f96"}
Jul 27 18:00:40.965977 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:40.965895    2565 generic.go:356] "Generic (PLEG): container finished" podID="04899f67-ccdb-4d1a-b678-aef97a978cce" containerID="a3dba95574fca3dc55b1e1bb3f14b54da5dd3767db1a781d33bee1e512a57673" exitCode=0
Jul 27 18:00:40.965977 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:40.965929    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79" event={"ID":"04899f67-ccdb-4d1a-b678-aef97a978cce","Type":"ContainerDied","Data":"a3dba95574fca3dc55b1e1bb3f14b54da5dd3767db1a781d33bee1e512a57673"}
Jul 27 18:00:42.097530 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:42.097506    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"
Jul 27 18:00:42.165234 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:42.165210    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04899f67-ccdb-4d1a-b678-aef97a978cce-maas-api-service-ca\") pod \"04899f67-ccdb-4d1a-b678-aef97a978cce\" (UID: \"04899f67-ccdb-4d1a-b678-aef97a978cce\") "
Jul 27 18:00:42.165546 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:42.165526    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04899f67-ccdb-4d1a-b678-aef97a978cce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "04899f67-ccdb-4d1a-b678-aef97a978cce" (UID: "04899f67-ccdb-4d1a-b678-aef97a978cce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:00:42.265898 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:42.265832    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04899f67-ccdb-4d1a-b678-aef97a978cce-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:00:42.976724 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:42.976696    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"
Jul 27 18:00:42.976724 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:42.976721    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79" event={"ID":"04899f67-ccdb-4d1a-b678-aef97a978cce","Type":"ContainerDied","Data":"08d728c5138310f95fe93d35dedab53f0de0c935f383696ad6cd806b85b34f96"}
Jul 27 18:00:42.976941 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:42.976761    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08d728c5138310f95fe93d35dedab53f0de0c935f383696ad6cd806b85b34f96"
Jul 27 18:00:43.333496 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:43.333461    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:00:43.339050 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:43.339027    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:00:59.468263 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:59.468225    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:00:59.517871 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:59.517840    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"]
Jul 27 18:00:59.520265 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:00:59.520242    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0dcf85-tzv79"]
Jul 27 18:01:01.255084 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:01.255044    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04899f67-ccdb-4d1a-b678-aef97a978cce" path="/var/lib/kubelet/pods/04899f67-ccdb-4d1a-b678-aef97a978cce/volumes"
Jul 27 18:01:14.567151 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:14.567119    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:01:24.180278 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:24.180244    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:01:41.907928 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.907885    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"]
Jul 27 18:01:41.908462 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.908368    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04899f67-ccdb-4d1a-b678-aef97a978cce" containerName="revoke-keys"
Jul 27 18:01:41.908462 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.908386    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04899f67-ccdb-4d1a-b678-aef97a978cce" containerName="revoke-keys"
Jul 27 18:01:41.908580 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.908506    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="04899f67-ccdb-4d1a-b678-aef97a978cce" containerName="revoke-keys"
Jul 27 18:01:41.911638 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.911618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:41.924185 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.923319    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"default-dockercfg-7g9zj\""
Jul 27 18:01:41.924185 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.923426    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"kube-root-ca.crt\""
Jul 27 18:01:41.924185 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.923512    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"openshift-service-ca.crt\""
Jul 27 18:01:41.924185 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.923654    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"auth-test-model-1aa3ff-kserve-self-signed-certs\""
Jul 27 18:01:41.926500 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:41.926478    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"]
Jul 27 18:01:42.056429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.056396    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-model-cache\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.056594 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.056440    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-home\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.056594 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.056462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8j4k\" (UniqueName: \"kubernetes.io/projected/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-kube-api-access-v8j4k\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.056594 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.056572    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tmp-dir\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.056730 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.056608    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-dshm\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.056730 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.056634    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tls-certs\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.157985 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.157899    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tmp-dir\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.157985 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.157940    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-dshm\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.157985 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.157962    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tls-certs\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.157985 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.157983    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-model-cache\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.158342 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.158010    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-home\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.158342 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.158027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8j4k\" (UniqueName: \"kubernetes.io/projected/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-kube-api-access-v8j4k\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.158455 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.158431    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-model-cache\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.158500 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.158464    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-home\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.158553 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.158527    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tmp-dir\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.160404 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.160374    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-dshm\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.160696 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.160677    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tls-certs\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.169563 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.169542    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8j4k\" (UniqueName: \"kubernetes.io/projected/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-kube-api-access-v8j4k\") pod \"auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") " pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.227149 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.227117    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:42.363963 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:42.363937    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"]
Jul 27 18:01:42.366274 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:01:42.366246    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b135ec9_036d_4f91_a1cb_6f36f4a512ac.slice/crio-babb0384b687b2036d89ad200275a01c0f974e328b3f34f690b83b25e06917e1 WatchSource:0}: Error finding container babb0384b687b2036d89ad200275a01c0f974e328b3f34f690b83b25e06917e1: Status 404 returned error can't find the container with id babb0384b687b2036d89ad200275a01c0f974e328b3f34f690b83b25e06917e1
Jul 27 18:01:43.258410 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:43.258379    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5" event={"ID":"8b135ec9-036d-4f91-a1cb-6f36f4a512ac","Type":"ContainerStarted","Data":"262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785"}
Jul 27 18:01:43.258781 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:43.258417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5" event={"ID":"8b135ec9-036d-4f91-a1cb-6f36f4a512ac","Type":"ContainerStarted","Data":"babb0384b687b2036d89ad200275a01c0f974e328b3f34f690b83b25e06917e1"}
Jul 27 18:01:43.310783 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:43.310737    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5" podStartSLOduration=2.310721589 podStartE2EDuration="2.310721589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:01:43.309974093 +0000 UTC m=+2460.570949292" watchObservedRunningTime="2026-07-27 18:01:43.310721589 +0000 UTC m=+2460.571696787"
Jul 27 18:01:43.442745 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:43.442716    2565 scope.go:117] "RemoveContainer" containerID="b90fb0dda58e5859c10747544c14140dadf3b020cf3cb4c480075bddbf3bbaf9"
Jul 27 18:01:45.313022 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:45.312988    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:01:52.227882 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:52.227838    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:52.227882 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:52.227882    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:52.240466 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:52.240444    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:01:52.308239 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:01:52.308206    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:02:16.204403 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.204372    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"]
Jul 27 18:02:16.207771 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.207755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.210731 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.210710    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"kube-root-ca.crt\""
Jul 27 18:02:16.212438 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.212419    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"openshift-service-ca.crt\""
Jul 27 18:02:16.213775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.213752    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"auth-test-model-ed5396-kserve-self-signed-certs\""
Jul 27 18:02:16.213882 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.213799    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"default-dockercfg-7q8bh\""
Jul 27 18:02:16.235498 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.235473    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"]
Jul 27 18:02:16.239855 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.239831    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-dshm\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.239967 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.239861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-model-cache\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.239967 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.239882    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpt44\" (UniqueName: \"kubernetes.io/projected/b645ffcb-b466-4f69-827a-c7292450bcec-kube-api-access-jpt44\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.240085 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.239981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-home\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.240085 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.240040    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-tmp-dir\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.240174 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.240106    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b645ffcb-b466-4f69-827a-c7292450bcec-tls-certs\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.340572 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.340531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-tmp-dir\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.340775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.340587    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b645ffcb-b466-4f69-827a-c7292450bcec-tls-certs\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.340775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.340612    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-dshm\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.340775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.340626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-model-cache\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.340775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.340643    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpt44\" (UniqueName: \"kubernetes.io/projected/b645ffcb-b466-4f69-827a-c7292450bcec-kube-api-access-jpt44\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.340775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.340689    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-home\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.341054 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.341016    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-tmp-dir\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.341101 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.341059    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-model-cache\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.341101 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.341090    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-home\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.342879 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.342857    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-dshm\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.343134 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.343119    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b645ffcb-b466-4f69-827a-c7292450bcec-tls-certs\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.352946 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.352923    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpt44\" (UniqueName: \"kubernetes.io/projected/b645ffcb-b466-4f69-827a-c7292450bcec-kube-api-access-jpt44\") pod \"auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") " pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.516946 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.516863    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:16.655831 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:16.655801    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"]
Jul 27 18:02:16.656756 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:02:16.656732    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb645ffcb_b466_4f69_827a_c7292450bcec.slice/crio-946fbb84919e8565630eb8e24563b918f978f402b7826ee042a9087c40e8cad2 WatchSource:0}: Error finding container 946fbb84919e8565630eb8e24563b918f978f402b7826ee042a9087c40e8cad2: Status 404 returned error can't find the container with id 946fbb84919e8565630eb8e24563b918f978f402b7826ee042a9087c40e8cad2
Jul 27 18:02:17.415104 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:17.415072    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr" event={"ID":"b645ffcb-b466-4f69-827a-c7292450bcec","Type":"ContainerStarted","Data":"4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1"}
Jul 27 18:02:17.415104 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:17.415109    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr" event={"ID":"b645ffcb-b466-4f69-827a-c7292450bcec","Type":"ContainerStarted","Data":"946fbb84919e8565630eb8e24563b918f978f402b7826ee042a9087c40e8cad2"}
Jul 27 18:02:17.441451 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:17.441409    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr" podStartSLOduration=1.441395729 podStartE2EDuration="1.441395729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:02:17.439553408 +0000 UTC m=+2494.700528606" watchObservedRunningTime="2026-07-27 18:02:17.441395729 +0000 UTC m=+2494.702370927"
Jul 27 18:02:19.692656 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:19.692622    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:02:26.517446 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:26.517407    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:26.517446 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:26.517457    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:26.530359 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:26.530335    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:27.471245 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:27.471208    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:02:50.616450 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:02:50.616417    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:03:11.695506 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:03:11.695456    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:03:30.900800 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:03:30.900764    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:03:58.188522 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:03:58.188485    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:04:16.078757 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:04:16.078721    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:04:35.974268 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:04:35.974233    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:04:55.067903 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:04:55.067869    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:05:15.575175 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:05:15.575136    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:05:34.470146 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:05:34.470093    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:05:43.362103 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:05:43.362025    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:05:43.369642 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:05:43.369617    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:05:43.482603 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:05:43.482571    2565 scope.go:117] "RemoveContainer" containerID="fc2504a3c414c08d57532591d20940a4d4abd56c48d976db983dcde1dfaf0aab"
Jul 27 18:05:56.394907 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:05:56.394869    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:06:14.974615 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:06:14.974578    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:06:43.501036 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:06:43.501003    2565 scope.go:117] "RemoveContainer" containerID="a98a1f9c0e44fbca6a05ec96746aaecf00b7b3e27273a8970ee0af54df90e97b"
Jul 27 18:06:43.509277 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:06:43.509259    2565 scope.go:117] "RemoveContainer" containerID="a3dba95574fca3dc55b1e1bb3f14b54da5dd3767db1a781d33bee1e512a57673"
Jul 27 18:06:43.516417 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:06:43.516399    2565 scope.go:117] "RemoveContainer" containerID="71e8e0f7dc06dd320a0b3f9019c0f36d038f45d1549e52f4963f7307cb543e25"
Jul 27 18:06:49.564371 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:06:49.564337    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:06:54.343721 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:06:54.343689    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"]
Jul 27 18:06:55.257910 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:06:55.257877    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"]
Jul 27 18:07:04.778830 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.778746    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"]
Jul 27 18:07:04.782085 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.782064    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.785064 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.785039    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"sub-test-model-1aa3ff-kserve-self-signed-certs\""
Jul 27 18:07:04.799727 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.799707    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"]
Jul 27 18:07:04.872149 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.872115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-model-cache\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.872348 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.872169    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1678a9-abff-4e77-bb71-16b79dbbde11-tls-certs\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.872348 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.872206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-home\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.872348 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.872226    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-tmp-dir\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.872348 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.872248    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-dshm\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.872348 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.872270    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7dtq\" (UniqueName: \"kubernetes.io/projected/2c1678a9-abff-4e77-bb71-16b79dbbde11-kube-api-access-s7dtq\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.973603 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.973523    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1678a9-abff-4e77-bb71-16b79dbbde11-tls-certs\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.973603 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.973575    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-home\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.973603 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.973603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-tmp-dir\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.973845 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.973630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-dshm\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.973845 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.973654    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7dtq\" (UniqueName: \"kubernetes.io/projected/2c1678a9-abff-4e77-bb71-16b79dbbde11-kube-api-access-s7dtq\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.973845 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.973712    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-model-cache\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.974007 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.973981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-home\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.974101 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.974079    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-model-cache\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.974101 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.974094    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-tmp-dir\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.975905 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.975886    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-dshm\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.976187 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.976169    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1678a9-abff-4e77-bb71-16b79dbbde11-tls-certs\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:04.982028 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:04.982009    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7dtq\" (UniqueName: \"kubernetes.io/projected/2c1678a9-abff-4e77-bb71-16b79dbbde11-kube-api-access-s7dtq\") pod \"sub-test-model-1aa3ff-kserve-59df788b4c-kk667\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") " pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:05.090654 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:05.090620    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:05.229920 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:05.229894    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"]
Jul 27 18:07:05.232138 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:07:05.232104    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c1678a9_abff_4e77_bb71_16b79dbbde11.slice/crio-2891de06a48d913e19d5fe8b02b40ded8c5dd3f1406dbb9c7182c61c6373ed20 WatchSource:0}: Error finding container 2891de06a48d913e19d5fe8b02b40ded8c5dd3f1406dbb9c7182c61c6373ed20: Status 404 returned error can't find the container with id 2891de06a48d913e19d5fe8b02b40ded8c5dd3f1406dbb9c7182c61c6373ed20
Jul 27 18:07:05.764044 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:05.764007    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667" event={"ID":"2c1678a9-abff-4e77-bb71-16b79dbbde11","Type":"ContainerStarted","Data":"7d0db53432c769ca416d408c64015c78b576884b3969696d4bb89fd5b8875519"}
Jul 27 18:07:05.764044 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:05.764056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667" event={"ID":"2c1678a9-abff-4e77-bb71-16b79dbbde11","Type":"ContainerStarted","Data":"2891de06a48d913e19d5fe8b02b40ded8c5dd3f1406dbb9c7182c61c6373ed20"}
Jul 27 18:07:05.774456 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:05.774425    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:07:05.784765 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:05.784719    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667" podStartSLOduration=1.784705981 podStartE2EDuration="1.784705981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:07:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:07:05.784460464 +0000 UTC m=+2783.045435664" watchObservedRunningTime="2026-07-27 18:07:05.784705981 +0000 UTC m=+2783.045681179"
Jul 27 18:07:09.362523 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.362461    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5" podUID="8b135ec9-036d-4f91-a1cb-6f36f4a512ac" containerName="main" containerID="cri-o://262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785" gracePeriod=45
Jul 27 18:07:09.613717 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.613656    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:07:09.712955 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.712919    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tls-certs\") pod \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") "
Jul 27 18:07:09.713126 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713006    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-dshm\") pod \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") "
Jul 27 18:07:09.713126 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713033    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-model-cache\") pod \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") "
Jul 27 18:07:09.713126 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713070    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tmp-dir\") pod \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") "
Jul 27 18:07:09.713126 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713100    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-home\") pod \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") "
Jul 27 18:07:09.713126 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713117    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8j4k\" (UniqueName: \"kubernetes.io/projected/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-kube-api-access-v8j4k\") pod \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\" (UID: \"8b135ec9-036d-4f91-a1cb-6f36f4a512ac\") "
Jul 27 18:07:09.713414 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713323    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b135ec9-036d-4f91-a1cb-6f36f4a512ac" (UID: "8b135ec9-036d-4f91-a1cb-6f36f4a512ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:09.713466 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713402    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-home" (OuterVolumeSpecName: "home") pod "8b135ec9-036d-4f91-a1cb-6f36f4a512ac" (UID: "8b135ec9-036d-4f91-a1cb-6f36f4a512ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:09.713466 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.713416    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b135ec9-036d-4f91-a1cb-6f36f4a512ac" (UID: "8b135ec9-036d-4f91-a1cb-6f36f4a512ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:09.715337 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.715311    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-dshm" (OuterVolumeSpecName: "dshm") pod "8b135ec9-036d-4f91-a1cb-6f36f4a512ac" (UID: "8b135ec9-036d-4f91-a1cb-6f36f4a512ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:09.715337 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.715313    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b135ec9-036d-4f91-a1cb-6f36f4a512ac" (UID: "8b135ec9-036d-4f91-a1cb-6f36f4a512ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:07:09.715478 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.715338    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-kube-api-access-v8j4k" (OuterVolumeSpecName: "kube-api-access-v8j4k") pod "8b135ec9-036d-4f91-a1cb-6f36f4a512ac" (UID: "8b135ec9-036d-4f91-a1cb-6f36f4a512ac"). InnerVolumeSpecName "kube-api-access-v8j4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:07:09.782217 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.782179    2565 generic.go:356] "Generic (PLEG): container finished" podID="8b135ec9-036d-4f91-a1cb-6f36f4a512ac" containerID="262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785" exitCode=0
Jul 27 18:07:09.782446 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.782223    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"
Jul 27 18:07:09.782446 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.782264    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5" event={"ID":"8b135ec9-036d-4f91-a1cb-6f36f4a512ac","Type":"ContainerDied","Data":"262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785"}
Jul 27 18:07:09.782446 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.782329    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5" event={"ID":"8b135ec9-036d-4f91-a1cb-6f36f4a512ac","Type":"ContainerDied","Data":"babb0384b687b2036d89ad200275a01c0f974e328b3f34f690b83b25e06917e1"}
Jul 27 18:07:09.782446 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.782351    2565 scope.go:117] "RemoveContainer" containerID="262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785"
Jul 27 18:07:09.791282 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.791263    2565 scope.go:117] "RemoveContainer" containerID="262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785"
Jul 27 18:07:09.791589 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:07:09.791570    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785\": container with ID starting with 262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785 not found: ID does not exist" containerID="262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785"
Jul 27 18:07:09.791642 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.791602    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785"} err="failed to get container status \"262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785\": rpc error: code = NotFound desc = could not find container \"262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785\": container with ID starting with 262ed205dc30a6f487ab570fa85fee5332a14d309fec36bad7c462b02edcb785 not found: ID does not exist"
Jul 27 18:07:09.805339 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.805309    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"]
Jul 27 18:07:09.808978 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.808953    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/auth-test-model-1aa3ff-kserve-5c86f8b675-d52d5"]
Jul 27 18:07:09.814644 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.814626    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:09.814644 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.814649    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:09.814760 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.814658    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:09.814760 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.814666    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:09.814760 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.814674    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:09.814760 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:09.814682    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8j4k\" (UniqueName: \"kubernetes.io/projected/8b135ec9-036d-4f91-a1cb-6f36f4a512ac-kube-api-access-v8j4k\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:10.276449 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.276407    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr" podUID="b645ffcb-b466-4f69-827a-c7292450bcec" containerName="main" containerID="cri-o://4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1" gracePeriod=45
Jul 27 18:07:10.536909 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.536850    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:07:10.621602 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621571    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpt44\" (UniqueName: \"kubernetes.io/projected/b645ffcb-b466-4f69-827a-c7292450bcec-kube-api-access-jpt44\") pod \"b645ffcb-b466-4f69-827a-c7292450bcec\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") "
Jul 27 18:07:10.621602 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621604    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-tmp-dir\") pod \"b645ffcb-b466-4f69-827a-c7292450bcec\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") "
Jul 27 18:07:10.621829 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621624    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-dshm\") pod \"b645ffcb-b466-4f69-827a-c7292450bcec\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") "
Jul 27 18:07:10.621829 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621685    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-model-cache\") pod \"b645ffcb-b466-4f69-827a-c7292450bcec\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") "
Jul 27 18:07:10.621829 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621701    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b645ffcb-b466-4f69-827a-c7292450bcec-tls-certs\") pod \"b645ffcb-b466-4f69-827a-c7292450bcec\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") "
Jul 27 18:07:10.621829 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621737    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-home\") pod \"b645ffcb-b466-4f69-827a-c7292450bcec\" (UID: \"b645ffcb-b466-4f69-827a-c7292450bcec\") "
Jul 27 18:07:10.622014 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621880    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b645ffcb-b466-4f69-827a-c7292450bcec" (UID: "b645ffcb-b466-4f69-827a-c7292450bcec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:10.622014 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.621947    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-model-cache" (OuterVolumeSpecName: "model-cache") pod "b645ffcb-b466-4f69-827a-c7292450bcec" (UID: "b645ffcb-b466-4f69-827a-c7292450bcec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:10.622116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.622061    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-home" (OuterVolumeSpecName: "home") pod "b645ffcb-b466-4f69-827a-c7292450bcec" (UID: "b645ffcb-b466-4f69-827a-c7292450bcec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:10.623821 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.623792    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-dshm" (OuterVolumeSpecName: "dshm") pod "b645ffcb-b466-4f69-827a-c7292450bcec" (UID: "b645ffcb-b466-4f69-827a-c7292450bcec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:07:10.623940 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.623859    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b645ffcb-b466-4f69-827a-c7292450bcec-kube-api-access-jpt44" (OuterVolumeSpecName: "kube-api-access-jpt44") pod "b645ffcb-b466-4f69-827a-c7292450bcec" (UID: "b645ffcb-b466-4f69-827a-c7292450bcec"). InnerVolumeSpecName "kube-api-access-jpt44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:07:10.623940 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.623892    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b645ffcb-b466-4f69-827a-c7292450bcec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b645ffcb-b466-4f69-827a-c7292450bcec" (UID: "b645ffcb-b466-4f69-827a-c7292450bcec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:07:10.722644 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.722614    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:10.722644 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.722638    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpt44\" (UniqueName: \"kubernetes.io/projected/b645ffcb-b466-4f69-827a-c7292450bcec-kube-api-access-jpt44\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:10.722644 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.722649    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:10.722868 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.722657    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:10.722868 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.722666    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b645ffcb-b466-4f69-827a-c7292450bcec-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:10.722868 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.722674    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b645ffcb-b466-4f69-827a-c7292450bcec-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:07:10.789364 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.789255    2565 generic.go:356] "Generic (PLEG): container finished" podID="b645ffcb-b466-4f69-827a-c7292450bcec" containerID="4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1" exitCode=0
Jul 27 18:07:10.789364 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.789320    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr" event={"ID":"b645ffcb-b466-4f69-827a-c7292450bcec","Type":"ContainerDied","Data":"4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1"}
Jul 27 18:07:10.789364 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.789356    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr" event={"ID":"b645ffcb-b466-4f69-827a-c7292450bcec","Type":"ContainerDied","Data":"946fbb84919e8565630eb8e24563b918f978f402b7826ee042a9087c40e8cad2"}
Jul 27 18:07:10.789588 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.789373    2565 scope.go:117] "RemoveContainer" containerID="4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1"
Jul 27 18:07:10.789588 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.789323    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"
Jul 27 18:07:10.806868 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.806847    2565 scope.go:117] "RemoveContainer" containerID="4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1"
Jul 27 18:07:10.807152 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:07:10.807131    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1\": container with ID starting with 4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1 not found: ID does not exist" containerID="4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1"
Jul 27 18:07:10.807219 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.807160    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1"} err="failed to get container status \"4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1\": rpc error: code = NotFound desc = could not find container \"4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1\": container with ID starting with 4d5dc7a85935a9eb7b66436381bf246e2896cdd4cfc4b739d9f1fca6d113bfb1 not found: ID does not exist"
Jul 27 18:07:10.817169 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.817142    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"]
Jul 27 18:07:10.821985 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:10.821966    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/auth-test-model-ed5396-kserve-5b9c7d4579-mfmnr"]
Jul 27 18:07:11.255445 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:11.255408    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b135ec9-036d-4f91-a1cb-6f36f4a512ac" path="/var/lib/kubelet/pods/8b135ec9-036d-4f91-a1cb-6f36f4a512ac/volumes"
Jul 27 18:07:11.255799 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:11.255786    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b645ffcb-b466-4f69-827a-c7292450bcec" path="/var/lib/kubelet/pods/b645ffcb-b466-4f69-827a-c7292450bcec/volumes"
Jul 27 18:07:15.091017 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:15.090988    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:15.091495 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:15.091050    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:15.103268 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:15.103247    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:15.472615 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:15.472534    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:07:15.823561 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:15.823528    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:07:27.965885 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.965848    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"]
Jul 27 18:07:27.966479 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.966181    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b135ec9-036d-4f91-a1cb-6f36f4a512ac" containerName="main"
Jul 27 18:07:27.966479 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.966192    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b135ec9-036d-4f91-a1cb-6f36f4a512ac" containerName="main"
Jul 27 18:07:27.966479 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.966206    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b645ffcb-b466-4f69-827a-c7292450bcec" containerName="main"
Jul 27 18:07:27.966479 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.966212    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b645ffcb-b466-4f69-827a-c7292450bcec" containerName="main"
Jul 27 18:07:27.966479 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.966282    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b135ec9-036d-4f91-a1cb-6f36f4a512ac" containerName="main"
Jul 27 18:07:27.966479 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.966308    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b645ffcb-b466-4f69-827a-c7292450bcec" containerName="main"
Jul 27 18:07:27.969211 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.969194    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:27.973680 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.973658    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"default-dockercfg-7q8bh\""
Jul 27 18:07:27.973813 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.973685    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"sub-test-model-ed5396-kserve-self-signed-certs\""
Jul 27 18:07:27.973813 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.973666    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"openshift-service-ca.crt\""
Jul 27 18:07:27.973813 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.973753    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"kube-root-ca.crt\""
Jul 27 18:07:27.984107 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:27.984088    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"]
Jul 27 18:07:28.069953 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.069921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-model-cache\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.070112 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.069957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-tmp-dir\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.070112 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.069985    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6270855-5d02-426b-97ce-838e6b0914d8-tls-certs\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.070112 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.070051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-dshm\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.070236 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.070111    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-home\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.070236 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.070136    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sswg5\" (UniqueName: \"kubernetes.io/projected/c6270855-5d02-426b-97ce-838e6b0914d8-kube-api-access-sswg5\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171374 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171324    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-dshm\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171374 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171382    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-home\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sswg5\" (UniqueName: \"kubernetes.io/projected/c6270855-5d02-426b-97ce-838e6b0914d8-kube-api-access-sswg5\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171445    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-model-cache\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171475    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-tmp-dir\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6270855-5d02-426b-97ce-838e6b0914d8-tls-certs\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171868 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-home\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.171911 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171887    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-tmp-dir\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.172016 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.171850    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-model-cache\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.178452 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.174041    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-dshm\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.178452 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.174801    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6270855-5d02-426b-97ce-838e6b0914d8-tls-certs\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.180334 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.180288    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sswg5\" (UniqueName: \"kubernetes.io/projected/c6270855-5d02-426b-97ce-838e6b0914d8-kube-api-access-sswg5\") pod \"sub-test-model-ed5396-kserve-79c8c575ff-svdqq\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") " pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.277836 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.277755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:28.410492 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.410466    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"]
Jul 27 18:07:28.412446 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:07:28.412418    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6270855_5d02_426b_97ce_838e6b0914d8.slice/crio-81e0ae261bad9909fba84bbb90dfed63feff7e397efad3e176377c675e721754 WatchSource:0}: Error finding container 81e0ae261bad9909fba84bbb90dfed63feff7e397efad3e176377c675e721754: Status 404 returned error can't find the container with id 81e0ae261bad9909fba84bbb90dfed63feff7e397efad3e176377c675e721754
Jul 27 18:07:28.872336 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.872281    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq" event={"ID":"c6270855-5d02-426b-97ce-838e6b0914d8","Type":"ContainerStarted","Data":"814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135"}
Jul 27 18:07:28.872506 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.872342    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq" event={"ID":"c6270855-5d02-426b-97ce-838e6b0914d8","Type":"ContainerStarted","Data":"81e0ae261bad9909fba84bbb90dfed63feff7e397efad3e176377c675e721754"}
Jul 27 18:07:28.893652 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:28.893604    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq" podStartSLOduration=1.893590654 podStartE2EDuration="1.893590654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:07:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:07:28.891724163 +0000 UTC m=+2806.152699355" watchObservedRunningTime="2026-07-27 18:07:28.893590654 +0000 UTC m=+2806.154565853"
Jul 27 18:07:31.666725 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:31.666689    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:07:38.278458 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:38.278420    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:38.278458 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:38.278469    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:38.291047 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:38.291014    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:07:38.928507 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:07:38.928470    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:08:03.874499 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:03.874461    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:08:24.481915 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:24.481880    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:08:29.953922 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:29.953890    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"]
Jul 27 18:08:30.773288 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:30.773251    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"]
Jul 27 18:08:41.761467 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:41.761392    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:08:44.972275 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:44.972234    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667" podUID="2c1678a9-abff-4e77-bb71-16b79dbbde11" containerName="main" containerID="cri-o://7d0db53432c769ca416d408c64015c78b576884b3969696d4bb89fd5b8875519" gracePeriod=45
Jul 27 18:08:45.213285 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.213258    2565 generic.go:356] "Generic (PLEG): container finished" podID="2c1678a9-abff-4e77-bb71-16b79dbbde11" containerID="7d0db53432c769ca416d408c64015c78b576884b3969696d4bb89fd5b8875519" exitCode=0
Jul 27 18:08:45.213436 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.213346    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667" event={"ID":"2c1678a9-abff-4e77-bb71-16b79dbbde11","Type":"ContainerDied","Data":"7d0db53432c769ca416d408c64015c78b576884b3969696d4bb89fd5b8875519"}
Jul 27 18:08:45.213436 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.213383    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667" event={"ID":"2c1678a9-abff-4e77-bb71-16b79dbbde11","Type":"ContainerDied","Data":"2891de06a48d913e19d5fe8b02b40ded8c5dd3f1406dbb9c7182c61c6373ed20"}
Jul 27 18:08:45.213436 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.213395    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2891de06a48d913e19d5fe8b02b40ded8c5dd3f1406dbb9c7182c61c6373ed20"
Jul 27 18:08:45.218681 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.218660    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:08:45.312954 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.312927    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1678a9-abff-4e77-bb71-16b79dbbde11-tls-certs\") pod \"2c1678a9-abff-4e77-bb71-16b79dbbde11\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") "
Jul 27 18:08:45.313128 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.312966    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-model-cache\") pod \"2c1678a9-abff-4e77-bb71-16b79dbbde11\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") "
Jul 27 18:08:45.313128 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.312991    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-tmp-dir\") pod \"2c1678a9-abff-4e77-bb71-16b79dbbde11\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") "
Jul 27 18:08:45.313128 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313042    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-home\") pod \"2c1678a9-abff-4e77-bb71-16b79dbbde11\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") "
Jul 27 18:08:45.313128 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313087    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-dshm\") pod \"2c1678a9-abff-4e77-bb71-16b79dbbde11\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") "
Jul 27 18:08:45.313128 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313126    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7dtq\" (UniqueName: \"kubernetes.io/projected/2c1678a9-abff-4e77-bb71-16b79dbbde11-kube-api-access-s7dtq\") pod \"2c1678a9-abff-4e77-bb71-16b79dbbde11\" (UID: \"2c1678a9-abff-4e77-bb71-16b79dbbde11\") "
Jul 27 18:08:45.313437 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313231    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c1678a9-abff-4e77-bb71-16b79dbbde11" (UID: "2c1678a9-abff-4e77-bb71-16b79dbbde11"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:45.313437 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313334    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c1678a9-abff-4e77-bb71-16b79dbbde11" (UID: "2c1678a9-abff-4e77-bb71-16b79dbbde11"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:45.313437 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313343    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-home" (OuterVolumeSpecName: "home") pod "2c1678a9-abff-4e77-bb71-16b79dbbde11" (UID: "2c1678a9-abff-4e77-bb71-16b79dbbde11"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:45.316351 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313874    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:45.316351 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313913    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:45.316351 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.313927    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:45.316832 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.316721    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c1678a9-abff-4e77-bb71-16b79dbbde11-kube-api-access-s7dtq" (OuterVolumeSpecName: "kube-api-access-s7dtq") pod "2c1678a9-abff-4e77-bb71-16b79dbbde11" (UID: "2c1678a9-abff-4e77-bb71-16b79dbbde11"). InnerVolumeSpecName "kube-api-access-s7dtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:08:45.317916 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.317074    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c1678a9-abff-4e77-bb71-16b79dbbde11-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c1678a9-abff-4e77-bb71-16b79dbbde11" (UID: "2c1678a9-abff-4e77-bb71-16b79dbbde11"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:08:45.321553 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.321528    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-dshm" (OuterVolumeSpecName: "dshm") pod "2c1678a9-abff-4e77-bb71-16b79dbbde11" (UID: "2c1678a9-abff-4e77-bb71-16b79dbbde11"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:45.414854 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.414817    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1678a9-abff-4e77-bb71-16b79dbbde11-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:45.414854 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.414846    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7dtq\" (UniqueName: \"kubernetes.io/projected/2c1678a9-abff-4e77-bb71-16b79dbbde11-kube-api-access-s7dtq\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:45.414854 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.414857    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1678a9-abff-4e77-bb71-16b79dbbde11-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:45.791780 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:45.791684    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq" podUID="c6270855-5d02-426b-97ce-838e6b0914d8" containerName="main" containerID="cri-o://814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135" gracePeriod=45
Jul 27 18:08:46.045843 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.045781    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:08:46.120984 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.120949    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-tmp-dir\") pod \"c6270855-5d02-426b-97ce-838e6b0914d8\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") "
Jul 27 18:08:46.121187 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.120995    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-dshm\") pod \"c6270855-5d02-426b-97ce-838e6b0914d8\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") "
Jul 27 18:08:46.121187 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.121027    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-model-cache\") pod \"c6270855-5d02-426b-97ce-838e6b0914d8\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") "
Jul 27 18:08:46.121187 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.121055    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sswg5\" (UniqueName: \"kubernetes.io/projected/c6270855-5d02-426b-97ce-838e6b0914d8-kube-api-access-sswg5\") pod \"c6270855-5d02-426b-97ce-838e6b0914d8\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") "
Jul 27 18:08:46.121187 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.121074    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-home\") pod \"c6270855-5d02-426b-97ce-838e6b0914d8\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") "
Jul 27 18:08:46.121187 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.121104    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6270855-5d02-426b-97ce-838e6b0914d8-tls-certs\") pod \"c6270855-5d02-426b-97ce-838e6b0914d8\" (UID: \"c6270855-5d02-426b-97ce-838e6b0914d8\") "
Jul 27 18:08:46.121497 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.121342    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6270855-5d02-426b-97ce-838e6b0914d8" (UID: "c6270855-5d02-426b-97ce-838e6b0914d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:46.121497 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.121379    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-home" (OuterVolumeSpecName: "home") pod "c6270855-5d02-426b-97ce-838e6b0914d8" (UID: "c6270855-5d02-426b-97ce-838e6b0914d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:46.121497 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.121397    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6270855-5d02-426b-97ce-838e6b0914d8" (UID: "c6270855-5d02-426b-97ce-838e6b0914d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:46.123217 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.123189    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-dshm" (OuterVolumeSpecName: "dshm") pod "c6270855-5d02-426b-97ce-838e6b0914d8" (UID: "c6270855-5d02-426b-97ce-838e6b0914d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:08:46.123389 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.123305    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6270855-5d02-426b-97ce-838e6b0914d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6270855-5d02-426b-97ce-838e6b0914d8" (UID: "c6270855-5d02-426b-97ce-838e6b0914d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:08:46.123389 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.123305    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6270855-5d02-426b-97ce-838e6b0914d8-kube-api-access-sswg5" (OuterVolumeSpecName: "kube-api-access-sswg5") pod "c6270855-5d02-426b-97ce-838e6b0914d8" (UID: "c6270855-5d02-426b-97ce-838e6b0914d8"). InnerVolumeSpecName "kube-api-access-sswg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:08:46.218638 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.218605    2565 generic.go:356] "Generic (PLEG): container finished" podID="c6270855-5d02-426b-97ce-838e6b0914d8" containerID="814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135" exitCode=0
Jul 27 18:08:46.218791 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.218651    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"
Jul 27 18:08:46.218791 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.218691    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq" event={"ID":"c6270855-5d02-426b-97ce-838e6b0914d8","Type":"ContainerDied","Data":"814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135"}
Jul 27 18:08:46.218791 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.218738    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq" event={"ID":"c6270855-5d02-426b-97ce-838e6b0914d8","Type":"ContainerDied","Data":"81e0ae261bad9909fba84bbb90dfed63feff7e397efad3e176377c675e721754"}
Jul 27 18:08:46.218791 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.218759    2565 scope.go:117] "RemoveContainer" containerID="814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135"
Jul 27 18:08:46.218967 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.218882    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"
Jul 27 18:08:46.221880 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.221856    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6270855-5d02-426b-97ce-838e6b0914d8-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:46.221880 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.221877    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:46.221880 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.221885    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:46.222109 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.221894    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:46.222109 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.221904    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sswg5\" (UniqueName: \"kubernetes.io/projected/c6270855-5d02-426b-97ce-838e6b0914d8-kube-api-access-sswg5\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:46.222109 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.221912    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6270855-5d02-426b-97ce-838e6b0914d8-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:08:46.228577 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.228555    2565 scope.go:117] "RemoveContainer" containerID="814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135"
Jul 27 18:08:46.228863 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:08:46.228835    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135\": container with ID starting with 814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135 not found: ID does not exist" containerID="814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135"
Jul 27 18:08:46.228943 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.228874    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135"} err="failed to get container status \"814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135\": rpc error: code = NotFound desc = could not find container \"814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135\": container with ID starting with 814e0044596d126ba1317b9305526eec37eb115b28bc3371be3c328a3b807135 not found: ID does not exist"
Jul 27 18:08:46.257523 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.257492    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"]
Jul 27 18:08:46.264197 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.264175    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/sub-test-model-1aa3ff-kserve-59df788b4c-kk667"]
Jul 27 18:08:46.283613 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.283587    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"]
Jul 27 18:08:46.291520 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:46.291501    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/sub-test-model-ed5396-kserve-79c8c575ff-svdqq"]
Jul 27 18:08:47.256236 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:47.256205    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c1678a9-abff-4e77-bb71-16b79dbbde11" path="/var/lib/kubelet/pods/2c1678a9-abff-4e77-bb71-16b79dbbde11/volumes"
Jul 27 18:08:47.256622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:47.256607    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6270855-5d02-426b-97ce-838e6b0914d8" path="/var/lib/kubelet/pods/c6270855-5d02-426b-97ce-838e6b0914d8/volumes"
Jul 27 18:08:50.481709 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.481680    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"]
Jul 27 18:08:50.482096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.482006    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c1678a9-abff-4e77-bb71-16b79dbbde11" containerName="main"
Jul 27 18:08:50.482096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.482018    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c1678a9-abff-4e77-bb71-16b79dbbde11" containerName="main"
Jul 27 18:08:50.482096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.482030    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6270855-5d02-426b-97ce-838e6b0914d8" containerName="main"
Jul 27 18:08:50.482096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.482036    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6270855-5d02-426b-97ce-838e6b0914d8" containerName="main"
Jul 27 18:08:50.482096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.482091    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6270855-5d02-426b-97ce-838e6b0914d8" containerName="main"
Jul 27 18:08:50.482096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.482098    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c1678a9-abff-4e77-bb71-16b79dbbde11" containerName="main"
Jul 27 18:08:50.486247 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.486233    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.489996 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.489966    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"kube-root-ca.crt\""
Jul 27 18:08:50.491372 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.491346    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"default-dockercfg-7g9zj\""
Jul 27 18:08:50.491615 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.491593    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"rate-test-model-1aa3ff-kserve-self-signed-certs\""
Jul 27 18:08:50.491747 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.491640    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1aa3ff\"/\"openshift-service-ca.crt\""
Jul 27 18:08:50.496994 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.496970    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"]
Jul 27 18:08:50.552663 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.552634    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-home\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.552815 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.552667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tls-certs\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.552815 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.552694    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn4xr\" (UniqueName: \"kubernetes.io/projected/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-kube-api-access-xn4xr\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.552815 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.552768    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-dshm\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.552815 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.552792    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-model-cache\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.552815 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.552808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tmp-dir\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653277 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653228    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-dshm\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653277 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653277    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-model-cache\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653277 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653311    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tmp-dir\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653582 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653439    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-home\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653582 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653481    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tls-certs\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653582 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn4xr\" (UniqueName: \"kubernetes.io/projected/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-kube-api-access-xn4xr\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653725 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653690    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tmp-dir\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653780 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653729    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-model-cache\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.653780 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.653770    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-home\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.655539 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.655519    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-dshm\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.655802 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.655786    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tls-certs\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.662830 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.662805    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn4xr\" (UniqueName: \"kubernetes.io/projected/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-kube-api-access-xn4xr\") pod \"rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") " pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.797606 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.797517    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:08:50.929336 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:50.929266    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"]
Jul 27 18:08:50.931565 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:08:50.931533    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e676278_5ccb_4ff0_9dcb_25a8b86627f1.slice/crio-dffd17b3917d65da701fda774e457117484b56918f5d382a783d269826e8a7c5 WatchSource:0}: Error finding container dffd17b3917d65da701fda774e457117484b56918f5d382a783d269826e8a7c5: Status 404 returned error can't find the container with id dffd17b3917d65da701fda774e457117484b56918f5d382a783d269826e8a7c5
Jul 27 18:08:51.243102 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:51.243054    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm" event={"ID":"2e676278-5ccb-4ff0-9dcb-25a8b86627f1","Type":"ContainerStarted","Data":"7af7b5bcfc4b352295d0382fab1fae16929524c5551387831d018ebac2db9f4f"}
Jul 27 18:08:51.243271 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:51.243110    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm" event={"ID":"2e676278-5ccb-4ff0-9dcb-25a8b86627f1","Type":"ContainerStarted","Data":"dffd17b3917d65da701fda774e457117484b56918f5d382a783d269826e8a7c5"}
Jul 27 18:08:51.266267 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:51.266192    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm" podStartSLOduration=1.266171343 podStartE2EDuration="1.266171343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:08:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:08:51.262510837 +0000 UTC m=+2888.523486034" watchObservedRunningTime="2026-07-27 18:08:51.266171343 +0000 UTC m=+2888.527146543"
Jul 27 18:08:53.983072 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:08:53.983036    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:09:00.798623 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:00.798592    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:09:00.799031 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:00.798636    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:09:00.811404 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:00.811378    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:09:01.299074 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:01.299047    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:09:40.128351 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.128319    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"]
Jul 27 18:09:40.131638 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.131621    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.142489 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.142469    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"openshift-service-ca.crt\""
Jul 27 18:09:40.143017 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.142999    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"rate-test-model-ed5396-kserve-self-signed-certs\""
Jul 27 18:09:40.143113 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.143019    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"default-dockercfg-7q8bh\""
Jul 27 18:09:40.143113 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.143049    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed5396\"/\"kube-root-ca.crt\""
Jul 27 18:09:40.155940 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.155916    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"]
Jul 27 18:09:40.278565 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.278522    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-tmp-dir\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.278565 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.278569    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-model-cache\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.278787 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.278653    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-home\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.278787 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.278693    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-dshm\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.278787 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.278749    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w2gh\" (UniqueName: \"kubernetes.io/projected/1a8e9814-3cfa-4070-9e9b-09f00b419722-kube-api-access-4w2gh\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.278884 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.278818    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8e9814-3cfa-4070-9e9b-09f00b419722-tls-certs\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380052 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.379961    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-home\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380052 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380000    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-dshm\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380052 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380052    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2gh\" (UniqueName: \"kubernetes.io/projected/1a8e9814-3cfa-4070-9e9b-09f00b419722-kube-api-access-4w2gh\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380395 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380072    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8e9814-3cfa-4070-9e9b-09f00b419722-tls-certs\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380395 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380090    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-tmp-dir\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380395 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380110    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-model-cache\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380565 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380400    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-home\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380565 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380503    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-tmp-dir\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.380565 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.380549    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-model-cache\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.382570 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.382544    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-dshm\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.382767 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.382750    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8e9814-3cfa-4070-9e9b-09f00b419722-tls-certs\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.393866 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.393844    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w2gh\" (UniqueName: \"kubernetes.io/projected/1a8e9814-3cfa-4070-9e9b-09f00b419722-kube-api-access-4w2gh\") pod \"rate-test-model-ed5396-kserve-5b485468b7-crs4c\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") " pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.441337 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.441289    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:40.578808 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:40.578775    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"]
Jul 27 18:09:40.582080 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:09:40.582050    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a8e9814_3cfa_4070_9e9b_09f00b419722.slice/crio-2b6f8313c25257e85679a9ff49192c77b2f1472f6779d7d26b66f8024bb31f5d WatchSource:0}: Error finding container 2b6f8313c25257e85679a9ff49192c77b2f1472f6779d7d26b66f8024bb31f5d: Status 404 returned error can't find the container with id 2b6f8313c25257e85679a9ff49192c77b2f1472f6779d7d26b66f8024bb31f5d
Jul 27 18:09:41.480758 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:41.480720    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c" event={"ID":"1a8e9814-3cfa-4070-9e9b-09f00b419722","Type":"ContainerStarted","Data":"0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26"}
Jul 27 18:09:41.480758 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:41.480762    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c" event={"ID":"1a8e9814-3cfa-4070-9e9b-09f00b419722","Type":"ContainerStarted","Data":"2b6f8313c25257e85679a9ff49192c77b2f1472f6779d7d26b66f8024bb31f5d"}
Jul 27 18:09:41.504020 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:41.503969    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c" podStartSLOduration=1.503955315 podStartE2EDuration="1.503955315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:09:41.503225417 +0000 UTC m=+2938.764200616" watchObservedRunningTime="2026-07-27 18:09:41.503955315 +0000 UTC m=+2938.764930514"
Jul 27 18:09:43.987623 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:43.987538    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:09:50.442103 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:50.442069    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:50.442592 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:50.442117    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:50.454402 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:50.454379    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:09:50.532600 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:09:50.532575    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:10:14.665063 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:14.665027    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:10:30.063088 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:30.063049    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:10:43.389725 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:43.389695    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:10:43.398390 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:43.398354    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:10:44.667871 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:44.667834    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:10:49.642428 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:49.642397    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"]
Jul 27 18:10:50.572108 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:50.572072    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"]
Jul 27 18:10:51.494806 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.494773    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140051076146608"]
Jul 27 18:10:51.498642 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.498610    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:51.501816 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.501791    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8fntq\""
Jul 27 18:10:51.505209 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.505185    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076146608"]
Jul 27 18:10:51.559787 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.559761    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82lzk\" (UniqueName: \"kubernetes.io/projected/ef148642-e79e-4f6a-afab-83a8e13670a4-kube-api-access-82lzk\") pod \"test-curl-7284-140051076146608\" (UID: \"ef148642-e79e-4f6a-afab-83a8e13670a4\") " pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:51.660943 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.660908    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82lzk\" (UniqueName: \"kubernetes.io/projected/ef148642-e79e-4f6a-afab-83a8e13670a4-kube-api-access-82lzk\") pod \"test-curl-7284-140051076146608\" (UID: \"ef148642-e79e-4f6a-afab-83a8e13670a4\") " pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:51.670066 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.670033    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82lzk\" (UniqueName: \"kubernetes.io/projected/ef148642-e79e-4f6a-afab-83a8e13670a4-kube-api-access-82lzk\") pod \"test-curl-7284-140051076146608\" (UID: \"ef148642-e79e-4f6a-afab-83a8e13670a4\") " pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:51.811439 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.811399    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:51.943630 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.943605    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076146608"]
Jul 27 18:10:51.946852 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:10:51.946817    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef148642_e79e_4f6a_afab_83a8e13670a4.slice/crio-bad3666c98dd7483029323aa061b0fb4709c9ba1d4081f52c8f6e076cccef29c WatchSource:0}: Error finding container bad3666c98dd7483029323aa061b0fb4709c9ba1d4081f52c8f6e076cccef29c: Status 404 returned error can't find the container with id bad3666c98dd7483029323aa061b0fb4709c9ba1d4081f52c8f6e076cccef29c
Jul 27 18:10:51.948308 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:51.948277    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 18:10:52.820269 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:52.820231    2565 generic.go:356] "Generic (PLEG): container finished" podID="ef148642-e79e-4f6a-afab-83a8e13670a4" containerID="174273467cd80166197928e2e3f53b2cd99877e7039fcd94d0103a164373a7f1" exitCode=0
Jul 27 18:10:52.820702 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:52.820355    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076146608" event={"ID":"ef148642-e79e-4f6a-afab-83a8e13670a4","Type":"ContainerDied","Data":"174273467cd80166197928e2e3f53b2cd99877e7039fcd94d0103a164373a7f1"}
Jul 27 18:10:52.820702 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:52.820423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076146608" event={"ID":"ef148642-e79e-4f6a-afab-83a8e13670a4","Type":"ContainerStarted","Data":"bad3666c98dd7483029323aa061b0fb4709c9ba1d4081f52c8f6e076cccef29c"}
Jul 27 18:10:53.960552 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:53.960527    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:53.980471 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:53.980442    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82lzk\" (UniqueName: \"kubernetes.io/projected/ef148642-e79e-4f6a-afab-83a8e13670a4-kube-api-access-82lzk\") pod \"ef148642-e79e-4f6a-afab-83a8e13670a4\" (UID: \"ef148642-e79e-4f6a-afab-83a8e13670a4\") "
Jul 27 18:10:53.982690 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:53.982660    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef148642-e79e-4f6a-afab-83a8e13670a4-kube-api-access-82lzk" (OuterVolumeSpecName: "kube-api-access-82lzk") pod "ef148642-e79e-4f6a-afab-83a8e13670a4" (UID: "ef148642-e79e-4f6a-afab-83a8e13670a4"). InnerVolumeSpecName "kube-api-access-82lzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:10:53.999505 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:53.999480    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140051076146608_ef148642-e79e-4f6a-afab-83a8e13670a4/test-curl-7284-140051076146608/0.log"
Jul 27 18:10:54.024131 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.024105    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140051076146608"]
Jul 27 18:10:54.027965 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.027944    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140051076146608"]
Jul 27 18:10:54.081973 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.081912    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82lzk\" (UniqueName: \"kubernetes.io/projected/ef148642-e79e-4f6a-afab-83a8e13670a4-kube-api-access-82lzk\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:10:54.144713 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.144687    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140051076198832"]
Jul 27 18:10:54.145023 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.145011    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef148642-e79e-4f6a-afab-83a8e13670a4" containerName="test-curl-7284-140051076146608"
Jul 27 18:10:54.145066 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.145025    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef148642-e79e-4f6a-afab-83a8e13670a4" containerName="test-curl-7284-140051076146608"
Jul 27 18:10:54.145114 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.145099    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef148642-e79e-4f6a-afab-83a8e13670a4" containerName="test-curl-7284-140051076146608"
Jul 27 18:10:54.147986 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.147971    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076198832"
Jul 27 18:10:54.156496 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.156476    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076198832"]
Jul 27 18:10:54.183107 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.183084    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdqv9\" (UniqueName: \"kubernetes.io/projected/9884a05d-6f9d-4ce6-8d1e-087913177963-kube-api-access-tdqv9\") pod \"test-curl-7284-140051076198832\" (UID: \"9884a05d-6f9d-4ce6-8d1e-087913177963\") " pod="opendatahub/test-curl-7284-140051076198832"
Jul 27 18:10:54.284106 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.284075    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdqv9\" (UniqueName: \"kubernetes.io/projected/9884a05d-6f9d-4ce6-8d1e-087913177963-kube-api-access-tdqv9\") pod \"test-curl-7284-140051076198832\" (UID: \"9884a05d-6f9d-4ce6-8d1e-087913177963\") " pod="opendatahub/test-curl-7284-140051076198832"
Jul 27 18:10:54.293100 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.293073    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdqv9\" (UniqueName: \"kubernetes.io/projected/9884a05d-6f9d-4ce6-8d1e-087913177963-kube-api-access-tdqv9\") pod \"test-curl-7284-140051076198832\" (UID: \"9884a05d-6f9d-4ce6-8d1e-087913177963\") " pod="opendatahub/test-curl-7284-140051076198832"
Jul 27 18:10:54.458648 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.458551    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076198832"
Jul 27 18:10:54.584927 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.584901    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076198832"]
Jul 27 18:10:54.587193 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:10:54.587162    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9884a05d_6f9d_4ce6_8d1e_087913177963.slice/crio-ba0fcae675ef9aa350270845d8061f562aafeb4ea8246d0f2607b6e3dc37d3a7 WatchSource:0}: Error finding container ba0fcae675ef9aa350270845d8061f562aafeb4ea8246d0f2607b6e3dc37d3a7: Status 404 returned error can't find the container with id ba0fcae675ef9aa350270845d8061f562aafeb4ea8246d0f2607b6e3dc37d3a7
Jul 27 18:10:54.830670 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.830633    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076198832" event={"ID":"9884a05d-6f9d-4ce6-8d1e-087913177963","Type":"ContainerStarted","Data":"ba0fcae675ef9aa350270845d8061f562aafeb4ea8246d0f2607b6e3dc37d3a7"}
Jul 27 18:10:54.831908 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.831881    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bad3666c98dd7483029323aa061b0fb4709c9ba1d4081f52c8f6e076cccef29c"
Jul 27 18:10:54.832029 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:54.831912    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:54.843230 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:10:54.843205    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7284-140051076146608\" is forbidden: User \"system:node:ip-10-0-131-8.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-8.ec2.internal' and this object" podUID="ef148642-e79e-4f6a-afab-83a8e13670a4" pod="opendatahub/test-curl-7284-140051076146608"
Jul 27 18:10:55.255491 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:55.255403    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef148642-e79e-4f6a-afab-83a8e13670a4" path="/var/lib/kubelet/pods/ef148642-e79e-4f6a-afab-83a8e13670a4/volumes"
Jul 27 18:10:55.843443 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:55.843405    2565 generic.go:356] "Generic (PLEG): container finished" podID="9884a05d-6f9d-4ce6-8d1e-087913177963" containerID="52d985d97d7cbf2286547e0d0c19437d1329d72e500572ff6ab1be317f9113dc" exitCode=0
Jul 27 18:10:55.843614 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:55.843442    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076198832" event={"ID":"9884a05d-6f9d-4ce6-8d1e-087913177963","Type":"ContainerDied","Data":"52d985d97d7cbf2286547e0d0c19437d1329d72e500572ff6ab1be317f9113dc"}
Jul 27 18:10:56.978256 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:56.978229    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076198832"
Jul 27 18:10:57.000034 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.000006    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140051076198832_9884a05d-6f9d-4ce6-8d1e-087913177963/test-curl-7284-140051076198832/0.log"
Jul 27 18:10:57.009726 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.009702    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdqv9\" (UniqueName: \"kubernetes.io/projected/9884a05d-6f9d-4ce6-8d1e-087913177963-kube-api-access-tdqv9\") pod \"9884a05d-6f9d-4ce6-8d1e-087913177963\" (UID: \"9884a05d-6f9d-4ce6-8d1e-087913177963\") "
Jul 27 18:10:57.011666 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.011640    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9884a05d-6f9d-4ce6-8d1e-087913177963-kube-api-access-tdqv9" (OuterVolumeSpecName: "kube-api-access-tdqv9") pod "9884a05d-6f9d-4ce6-8d1e-087913177963" (UID: "9884a05d-6f9d-4ce6-8d1e-087913177963"). InnerVolumeSpecName "kube-api-access-tdqv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:10:57.021434 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.021413    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140051076198832"]
Jul 27 18:10:57.025010 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.024987    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140051076198832"]
Jul 27 18:10:57.110806 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.110716    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdqv9\" (UniqueName: \"kubernetes.io/projected/9884a05d-6f9d-4ce6-8d1e-087913177963-kube-api-access-tdqv9\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:10:57.181413 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.181374    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:10:57.181777 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.181763    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9884a05d-6f9d-4ce6-8d1e-087913177963" containerName="test-curl-7284-140051076198832"
Jul 27 18:10:57.181838 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.181779    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9884a05d-6f9d-4ce6-8d1e-087913177963" containerName="test-curl-7284-140051076198832"
Jul 27 18:10:57.181876 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.181855    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9884a05d-6f9d-4ce6-8d1e-087913177963" containerName="test-curl-7284-140051076198832"
Jul 27 18:10:57.184864 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.184847    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:10:57.190604 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.190577    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:10:57.211930 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.211907    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpmsl\" (UniqueName: \"kubernetes.io/projected/5000c99d-a7f2-4c80-badb-752cbf87f77e-kube-api-access-vpmsl\") pod \"test-curl-7284-140051076179840\" (UID: \"5000c99d-a7f2-4c80-badb-752cbf87f77e\") " pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:10:57.255819 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.255791    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9884a05d-6f9d-4ce6-8d1e-087913177963" path="/var/lib/kubelet/pods/9884a05d-6f9d-4ce6-8d1e-087913177963/volumes"
Jul 27 18:10:57.312662 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.312636    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpmsl\" (UniqueName: \"kubernetes.io/projected/5000c99d-a7f2-4c80-badb-752cbf87f77e-kube-api-access-vpmsl\") pod \"test-curl-7284-140051076179840\" (UID: \"5000c99d-a7f2-4c80-badb-752cbf87f77e\") " pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:10:57.321200 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.321175    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpmsl\" (UniqueName: \"kubernetes.io/projected/5000c99d-a7f2-4c80-badb-752cbf87f77e-kube-api-access-vpmsl\") pod \"test-curl-7284-140051076179840\" (UID: \"5000c99d-a7f2-4c80-badb-752cbf87f77e\") " pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:10:57.495248 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.495172    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:10:57.621557 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.621516    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:10:57.623640 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:10:57.623615    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5000c99d_a7f2_4c80_badb_752cbf87f77e.slice/crio-63b72981e575065c9dd8ad82ef868c03386aafb5d3f36e1f4df2b952ed0969c9 WatchSource:0}: Error finding container 63b72981e575065c9dd8ad82ef868c03386aafb5d3f36e1f4df2b952ed0969c9: Status 404 returned error can't find the container with id 63b72981e575065c9dd8ad82ef868c03386aafb5d3f36e1f4df2b952ed0969c9
Jul 27 18:10:57.853287 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.853248    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076179840" event={"ID":"5000c99d-a7f2-4c80-badb-752cbf87f77e","Type":"ContainerStarted","Data":"63b72981e575065c9dd8ad82ef868c03386aafb5d3f36e1f4df2b952ed0969c9"}
Jul 27 18:10:57.854397 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.854375    2565 scope.go:117] "RemoveContainer" containerID="52d985d97d7cbf2286547e0d0c19437d1329d72e500572ff6ab1be317f9113dc"
Jul 27 18:10:57.854397 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:57.854394    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076198832"
Jul 27 18:10:58.861813 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:58.861780    2565 generic.go:356] "Generic (PLEG): container finished" podID="5000c99d-a7f2-4c80-badb-752cbf87f77e" containerID="3136e7bcccaad722593f54cd33b440b7c35ec8945828b8f8e6e2f7bb4aaf219d" exitCode=0
Jul 27 18:10:58.861813 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:58.861816    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076179840" event={"ID":"5000c99d-a7f2-4c80-badb-752cbf87f77e","Type":"ContainerDied","Data":"3136e7bcccaad722593f54cd33b440b7c35ec8945828b8f8e6e2f7bb4aaf219d"}
Jul 27 18:10:59.994975 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:10:59.994951    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:00.015982 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.015958    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140051076179840_5000c99d-a7f2-4c80-badb-752cbf87f77e/test-curl-7284-140051076179840/0.log"
Jul 27 18:11:00.038487 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.038448    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:11:00.038724 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.038708    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpmsl\" (UniqueName: \"kubernetes.io/projected/5000c99d-a7f2-4c80-badb-752cbf87f77e-kube-api-access-vpmsl\") pod \"5000c99d-a7f2-4c80-badb-752cbf87f77e\" (UID: \"5000c99d-a7f2-4c80-badb-752cbf87f77e\") "
Jul 27 18:11:00.041012 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.040981    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5000c99d-a7f2-4c80-badb-752cbf87f77e-kube-api-access-vpmsl" (OuterVolumeSpecName: "kube-api-access-vpmsl") pod "5000c99d-a7f2-4c80-badb-752cbf87f77e" (UID: "5000c99d-a7f2-4c80-badb-752cbf87f77e"). InnerVolumeSpecName "kube-api-access-vpmsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:11:00.042016 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.041994    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:11:00.140031 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.139941    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpmsl\" (UniqueName: \"kubernetes.io/projected/5000c99d-a7f2-4c80-badb-752cbf87f77e-kube-api-access-vpmsl\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:00.152819 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.152786    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:11:00.153122 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.153109    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5000c99d-a7f2-4c80-badb-752cbf87f77e" containerName="test-curl-7284-140051076179840"
Jul 27 18:11:00.153168 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.153123    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5000c99d-a7f2-4c80-badb-752cbf87f77e" containerName="test-curl-7284-140051076179840"
Jul 27 18:11:00.153210 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.153196    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5000c99d-a7f2-4c80-badb-752cbf87f77e" containerName="test-curl-7284-140051076179840"
Jul 27 18:11:00.156066 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.156049    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:00.163719 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.163690    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:11:00.241018 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.240978    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qqzw\" (UniqueName: \"kubernetes.io/projected/18e0eb09-b135-4811-a527-ca358181f72e-kube-api-access-6qqzw\") pod \"test-curl-7284-140051076179840\" (UID: \"18e0eb09-b135-4811-a527-ca358181f72e\") " pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:00.341649 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.341612    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qqzw\" (UniqueName: \"kubernetes.io/projected/18e0eb09-b135-4811-a527-ca358181f72e-kube-api-access-6qqzw\") pod \"test-curl-7284-140051076179840\" (UID: \"18e0eb09-b135-4811-a527-ca358181f72e\") " pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:00.350449 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.350410    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qqzw\" (UniqueName: \"kubernetes.io/projected/18e0eb09-b135-4811-a527-ca358181f72e-kube-api-access-6qqzw\") pod \"test-curl-7284-140051076179840\" (UID: \"18e0eb09-b135-4811-a527-ca358181f72e\") " pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:00.465571 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.465455    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:00.595419 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.595395    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:11:00.597872 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:11:00.597847    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e0eb09_b135_4811_a527_ca358181f72e.slice/crio-47c98e3116616ba7abbfa27fc7102ae2b6d4a2af3d4e04dc47963740a321f365 WatchSource:0}: Error finding container 47c98e3116616ba7abbfa27fc7102ae2b6d4a2af3d4e04dc47963740a321f365: Status 404 returned error can't find the container with id 47c98e3116616ba7abbfa27fc7102ae2b6d4a2af3d4e04dc47963740a321f365
Jul 27 18:11:00.872444 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.872411    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:00.872630 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.872448    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63b72981e575065c9dd8ad82ef868c03386aafb5d3f36e1f4df2b952ed0969c9"
Jul 27 18:11:00.873624 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.873597    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076179840" event={"ID":"18e0eb09-b135-4811-a527-ca358181f72e","Type":"ContainerStarted","Data":"47c98e3116616ba7abbfa27fc7102ae2b6d4a2af3d4e04dc47963740a321f365"}
Jul 27 18:11:00.886812 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:00.886782    2565 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7284-140051076179840" oldPodUID="5000c99d-a7f2-4c80-badb-752cbf87f77e" podUID="18e0eb09-b135-4811-a527-ca358181f72e"
Jul 27 18:11:01.195775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:01.195690    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:11:01.256745 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:01.256715    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5000c99d-a7f2-4c80-badb-752cbf87f77e" path="/var/lib/kubelet/pods/5000c99d-a7f2-4c80-badb-752cbf87f77e/volumes"
Jul 27 18:11:01.879675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:01.879639    2565 generic.go:356] "Generic (PLEG): container finished" podID="18e0eb09-b135-4811-a527-ca358181f72e" containerID="6660d69825157ff6143eb3a29373927a3e4792bbc9ad4ac852ac4c7e01845a25" exitCode=0
Jul 27 18:11:01.879859 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:01.879680    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076179840" event={"ID":"18e0eb09-b135-4811-a527-ca358181f72e","Type":"ContainerDied","Data":"6660d69825157ff6143eb3a29373927a3e4792bbc9ad4ac852ac4c7e01845a25"}
Jul 27 18:11:03.021578 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.021546    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:03.042942 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.042914    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140051076179840_18e0eb09-b135-4811-a527-ca358181f72e/test-curl-7284-140051076179840/0.log"
Jul 27 18:11:03.064175 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.064131    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:11:03.064412 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.064393    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qqzw\" (UniqueName: \"kubernetes.io/projected/18e0eb09-b135-4811-a527-ca358181f72e-kube-api-access-6qqzw\") pod \"18e0eb09-b135-4811-a527-ca358181f72e\" (UID: \"18e0eb09-b135-4811-a527-ca358181f72e\") "
Jul 27 18:11:03.066468 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.066437    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e0eb09-b135-4811-a527-ca358181f72e-kube-api-access-6qqzw" (OuterVolumeSpecName: "kube-api-access-6qqzw") pod "18e0eb09-b135-4811-a527-ca358181f72e" (UID: "18e0eb09-b135-4811-a527-ca358181f72e"). InnerVolumeSpecName "kube-api-access-6qqzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:11:03.068568 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.068545    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140051076179840"]
Jul 27 18:11:03.165674 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.165563    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qqzw\" (UniqueName: \"kubernetes.io/projected/18e0eb09-b135-4811-a527-ca358181f72e-kube-api-access-6qqzw\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:03.183713 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.183681    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:03.184016 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.184003    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e0eb09-b135-4811-a527-ca358181f72e" containerName="test-curl-7284-140051076179840"
Jul 27 18:11:03.184058 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.184017    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e0eb09-b135-4811-a527-ca358181f72e" containerName="test-curl-7284-140051076179840"
Jul 27 18:11:03.184100 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.184091    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e0eb09-b135-4811-a527-ca358181f72e" containerName="test-curl-7284-140051076179840"
Jul 27 18:11:03.186963 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.186946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:03.194993 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.194958    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:03.256152 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.256122    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e0eb09-b135-4811-a527-ca358181f72e" path="/var/lib/kubelet/pods/18e0eb09-b135-4811-a527-ca358181f72e/volumes"
Jul 27 18:11:03.266222 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.266200    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lrr8\" (UniqueName: \"kubernetes.io/projected/3e29f6b1-3156-460a-9e31-5aa55c283963-kube-api-access-7lrr8\") pod \"test-curl-7284-140051076097632\" (UID: \"3e29f6b1-3156-460a-9e31-5aa55c283963\") " pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:03.367738 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.367704    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lrr8\" (UniqueName: \"kubernetes.io/projected/3e29f6b1-3156-460a-9e31-5aa55c283963-kube-api-access-7lrr8\") pod \"test-curl-7284-140051076097632\" (UID: \"3e29f6b1-3156-460a-9e31-5aa55c283963\") " pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:03.377127 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.377099    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lrr8\" (UniqueName: \"kubernetes.io/projected/3e29f6b1-3156-460a-9e31-5aa55c283963-kube-api-access-7lrr8\") pod \"test-curl-7284-140051076097632\" (UID: \"3e29f6b1-3156-460a-9e31-5aa55c283963\") " pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:03.496216 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.496137    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:03.633060 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.633028    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:03.633256 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:11:03.633228    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e29f6b1_3156_460a_9e31_5aa55c283963.slice/crio-d1b0cf33ffc321bc856d009cc8d093444bbe446841bd65f6772dd749b5b150cb WatchSource:0}: Error finding container d1b0cf33ffc321bc856d009cc8d093444bbe446841bd65f6772dd749b5b150cb: Status 404 returned error can't find the container with id d1b0cf33ffc321bc856d009cc8d093444bbe446841bd65f6772dd749b5b150cb
Jul 27 18:11:03.890913 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.890874    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076097632" event={"ID":"3e29f6b1-3156-460a-9e31-5aa55c283963","Type":"ContainerStarted","Data":"d1b0cf33ffc321bc856d009cc8d093444bbe446841bd65f6772dd749b5b150cb"}
Jul 27 18:11:03.892068 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.892051    2565 scope.go:117] "RemoveContainer" containerID="6660d69825157ff6143eb3a29373927a3e4792bbc9ad4ac852ac4c7e01845a25"
Jul 27 18:11:03.892068 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:03.892065    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076179840"
Jul 27 18:11:04.660738 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.660653    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm" podUID="2e676278-5ccb-4ff0-9dcb-25a8b86627f1" containerName="main" containerID="cri-o://7af7b5bcfc4b352295d0382fab1fae16929524c5551387831d018ebac2db9f4f" gracePeriod=45
Jul 27 18:11:04.899917 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.899890    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e676278-5ccb-4ff0-9dcb-25a8b86627f1" containerID="7af7b5bcfc4b352295d0382fab1fae16929524c5551387831d018ebac2db9f4f" exitCode=0
Jul 27 18:11:04.900048 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.899958    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm" event={"ID":"2e676278-5ccb-4ff0-9dcb-25a8b86627f1","Type":"ContainerDied","Data":"7af7b5bcfc4b352295d0382fab1fae16929524c5551387831d018ebac2db9f4f"}
Jul 27 18:11:04.900048 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.900000    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm" event={"ID":"2e676278-5ccb-4ff0-9dcb-25a8b86627f1","Type":"ContainerDied","Data":"dffd17b3917d65da701fda774e457117484b56918f5d382a783d269826e8a7c5"}
Jul 27 18:11:04.900048 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.900020    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dffd17b3917d65da701fda774e457117484b56918f5d382a783d269826e8a7c5"
Jul 27 18:11:04.901516 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.901494    2565 generic.go:356] "Generic (PLEG): container finished" podID="3e29f6b1-3156-460a-9e31-5aa55c283963" containerID="987dff7130b61097b5c8f3818969f5c0e0729569a101ed3fa608bda2682445dd" exitCode=0
Jul 27 18:11:04.901639 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.901556    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076097632" event={"ID":"3e29f6b1-3156-460a-9e31-5aa55c283963","Type":"ContainerDied","Data":"987dff7130b61097b5c8f3818969f5c0e0729569a101ed3fa608bda2682445dd"}
Jul 27 18:11:04.905268 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.905245    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:11:04.982849 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.982784    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-home\") pod \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") "
Jul 27 18:11:04.982849 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.982836    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-model-cache\") pod \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") "
Jul 27 18:11:04.983089 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.982871    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn4xr\" (UniqueName: \"kubernetes.io/projected/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-kube-api-access-xn4xr\") pod \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") "
Jul 27 18:11:04.983089 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.982889    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tmp-dir\") pod \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") "
Jul 27 18:11:04.983089 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.982952    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-dshm\") pod \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") "
Jul 27 18:11:04.983089 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.982993    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tls-certs\") pod \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\" (UID: \"2e676278-5ccb-4ff0-9dcb-25a8b86627f1\") "
Jul 27 18:11:04.983089 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.983038    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-home" (OuterVolumeSpecName: "home") pod "2e676278-5ccb-4ff0-9dcb-25a8b86627f1" (UID: "2e676278-5ccb-4ff0-9dcb-25a8b86627f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:04.983371 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.983139    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e676278-5ccb-4ff0-9dcb-25a8b86627f1" (UID: "2e676278-5ccb-4ff0-9dcb-25a8b86627f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:04.983371 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.983222    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e676278-5ccb-4ff0-9dcb-25a8b86627f1" (UID: "2e676278-5ccb-4ff0-9dcb-25a8b86627f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:04.983371 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.983238    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:04.983371 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.983251    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:04.985044 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.985017    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-kube-api-access-xn4xr" (OuterVolumeSpecName: "kube-api-access-xn4xr") pod "2e676278-5ccb-4ff0-9dcb-25a8b86627f1" (UID: "2e676278-5ccb-4ff0-9dcb-25a8b86627f1"). InnerVolumeSpecName "kube-api-access-xn4xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:11:04.985044 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.985029    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e676278-5ccb-4ff0-9dcb-25a8b86627f1" (UID: "2e676278-5ccb-4ff0-9dcb-25a8b86627f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:11:04.985173 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:04.985039    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-dshm" (OuterVolumeSpecName: "dshm") pod "2e676278-5ccb-4ff0-9dcb-25a8b86627f1" (UID: "2e676278-5ccb-4ff0-9dcb-25a8b86627f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:05.084264 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.084240    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.084264 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.084263    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.084432 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.084273    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn4xr\" (UniqueName: \"kubernetes.io/projected/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-kube-api-access-xn4xr\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.084432 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.084283    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e676278-5ccb-4ff0-9dcb-25a8b86627f1-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.590713 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.590667    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c" podUID="1a8e9814-3cfa-4070-9e9b-09f00b419722" containerName="main" containerID="cri-o://0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26" gracePeriod=45
Jul 27 18:11:05.843911 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.843855    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:11:05.888196 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888166    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-model-cache\") pod \"1a8e9814-3cfa-4070-9e9b-09f00b419722\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") "
Jul 27 18:11:05.888391 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888209    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-dshm\") pod \"1a8e9814-3cfa-4070-9e9b-09f00b419722\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") "
Jul 27 18:11:05.888391 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888247    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-tmp-dir\") pod \"1a8e9814-3cfa-4070-9e9b-09f00b419722\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") "
Jul 27 18:11:05.888391 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888270    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4w2gh\" (UniqueName: \"kubernetes.io/projected/1a8e9814-3cfa-4070-9e9b-09f00b419722-kube-api-access-4w2gh\") pod \"1a8e9814-3cfa-4070-9e9b-09f00b419722\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") "
Jul 27 18:11:05.888391 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888317    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8e9814-3cfa-4070-9e9b-09f00b419722-tls-certs\") pod \"1a8e9814-3cfa-4070-9e9b-09f00b419722\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") "
Jul 27 18:11:05.888391 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888381    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-home\") pod \"1a8e9814-3cfa-4070-9e9b-09f00b419722\" (UID: \"1a8e9814-3cfa-4070-9e9b-09f00b419722\") "
Jul 27 18:11:05.888665 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888508    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a8e9814-3cfa-4070-9e9b-09f00b419722" (UID: "1a8e9814-3cfa-4070-9e9b-09f00b419722"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:05.888727 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888650    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a8e9814-3cfa-4070-9e9b-09f00b419722" (UID: "1a8e9814-3cfa-4070-9e9b-09f00b419722"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:05.888727 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888674    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.888844 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.888775    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-home" (OuterVolumeSpecName: "home") pod "1a8e9814-3cfa-4070-9e9b-09f00b419722" (UID: "1a8e9814-3cfa-4070-9e9b-09f00b419722"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:05.890695 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.890664    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a8e9814-3cfa-4070-9e9b-09f00b419722-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a8e9814-3cfa-4070-9e9b-09f00b419722" (UID: "1a8e9814-3cfa-4070-9e9b-09f00b419722"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:11:05.890695 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.890664    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-dshm" (OuterVolumeSpecName: "dshm") pod "1a8e9814-3cfa-4070-9e9b-09f00b419722" (UID: "1a8e9814-3cfa-4070-9e9b-09f00b419722"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:11:05.890966 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.890948    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8e9814-3cfa-4070-9e9b-09f00b419722-kube-api-access-4w2gh" (OuterVolumeSpecName: "kube-api-access-4w2gh") pod "1a8e9814-3cfa-4070-9e9b-09f00b419722" (UID: "1a8e9814-3cfa-4070-9e9b-09f00b419722"). InnerVolumeSpecName "kube-api-access-4w2gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:11:05.907332 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.907274    2565 generic.go:356] "Generic (PLEG): container finished" podID="1a8e9814-3cfa-4070-9e9b-09f00b419722" containerID="0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26" exitCode=0
Jul 27 18:11:05.907491 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.907347    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"
Jul 27 18:11:05.907491 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.907355    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c" event={"ID":"1a8e9814-3cfa-4070-9e9b-09f00b419722","Type":"ContainerDied","Data":"0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26"}
Jul 27 18:11:05.907491 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.907397    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c" event={"ID":"1a8e9814-3cfa-4070-9e9b-09f00b419722","Type":"ContainerDied","Data":"2b6f8313c25257e85679a9ff49192c77b2f1472f6779d7d26b66f8024bb31f5d"}
Jul 27 18:11:05.907491 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.907414    2565 scope.go:117] "RemoveContainer" containerID="0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26"
Jul 27 18:11:05.907699 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.907681    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"
Jul 27 18:11:05.916309 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.916270    2565 scope.go:117] "RemoveContainer" containerID="0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26"
Jul 27 18:11:05.916696 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:11:05.916675    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26\": container with ID starting with 0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26 not found: ID does not exist" containerID="0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26"
Jul 27 18:11:05.916788 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.916705    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26"} err="failed to get container status \"0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26\": rpc error: code = NotFound desc = could not find container \"0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26\": container with ID starting with 0ef64119f68dc1d36c635112b9b7ab23893f75c72e9cccacd5e4b677c76a2b26 not found: ID does not exist"
Jul 27 18:11:05.933216 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.933181    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"]
Jul 27 18:11:05.938115 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.938085    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed5396/rate-test-model-ed5396-kserve-5b485468b7-crs4c"]
Jul 27 18:11:05.953372 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.953349    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"]
Jul 27 18:11:05.961321 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.961279    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1aa3ff/rate-test-model-1aa3ff-kserve-78b4dbfd4c-8fqhm"]
Jul 27 18:11:05.989959 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.989933    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.990065 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.989989    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.990065 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.990006    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4w2gh\" (UniqueName: \"kubernetes.io/projected/1a8e9814-3cfa-4070-9e9b-09f00b419722-kube-api-access-4w2gh\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.990065 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.990021    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a8e9814-3cfa-4070-9e9b-09f00b419722-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:05.990065 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:05.990036    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a8e9814-3cfa-4070-9e9b-09f00b419722-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:06.046568 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.046544    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:06.069950 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.069928    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140051076097632_3e29f6b1-3156-460a-9e31-5aa55c283963/test-curl-7284-140051076097632/0.log"
Jul 27 18:11:06.090683 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.090548    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lrr8\" (UniqueName: \"kubernetes.io/projected/3e29f6b1-3156-460a-9e31-5aa55c283963-kube-api-access-7lrr8\") pod \"3e29f6b1-3156-460a-9e31-5aa55c283963\" (UID: \"3e29f6b1-3156-460a-9e31-5aa55c283963\") "
Jul 27 18:11:06.092385 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.092362    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:06.092775 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.092751    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e29f6b1-3156-460a-9e31-5aa55c283963-kube-api-access-7lrr8" (OuterVolumeSpecName: "kube-api-access-7lrr8") pod "3e29f6b1-3156-460a-9e31-5aa55c283963" (UID: "3e29f6b1-3156-460a-9e31-5aa55c283963"). InnerVolumeSpecName "kube-api-access-7lrr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:11:06.096915 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.096862    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:06.191756 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.191727    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lrr8\" (UniqueName: \"kubernetes.io/projected/3e29f6b1-3156-460a-9e31-5aa55c283963-kube-api-access-7lrr8\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:06.215164 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215135    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:06.215473 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215460    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8e9814-3cfa-4070-9e9b-09f00b419722" containerName="main"
Jul 27 18:11:06.215528 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215475    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8e9814-3cfa-4070-9e9b-09f00b419722" containerName="main"
Jul 27 18:11:06.215528 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215492    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e676278-5ccb-4ff0-9dcb-25a8b86627f1" containerName="main"
Jul 27 18:11:06.215528 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215498    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e676278-5ccb-4ff0-9dcb-25a8b86627f1" containerName="main"
Jul 27 18:11:06.215528 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215505    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e29f6b1-3156-460a-9e31-5aa55c283963" containerName="test-curl-7284-140051076097632"
Jul 27 18:11:06.215528 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215511    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e29f6b1-3156-460a-9e31-5aa55c283963" containerName="test-curl-7284-140051076097632"
Jul 27 18:11:06.215706 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215571    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e676278-5ccb-4ff0-9dcb-25a8b86627f1" containerName="main"
Jul 27 18:11:06.215706 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215579    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8e9814-3cfa-4070-9e9b-09f00b419722" containerName="main"
Jul 27 18:11:06.215706 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.215586    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e29f6b1-3156-460a-9e31-5aa55c283963" containerName="test-curl-7284-140051076097632"
Jul 27 18:11:06.218358 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.218343    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:06.225691 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.225650    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:06.292915 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.292883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrk4f\" (UniqueName: \"kubernetes.io/projected/43166304-9d14-4c0e-8c52-c207245348a4-kube-api-access-nrk4f\") pod \"test-curl-7284-140051076097632\" (UID: \"43166304-9d14-4c0e-8c52-c207245348a4\") " pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:06.394158 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.394069    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrk4f\" (UniqueName: \"kubernetes.io/projected/43166304-9d14-4c0e-8c52-c207245348a4-kube-api-access-nrk4f\") pod \"test-curl-7284-140051076097632\" (UID: \"43166304-9d14-4c0e-8c52-c207245348a4\") " pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:06.403514 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.403486    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrk4f\" (UniqueName: \"kubernetes.io/projected/43166304-9d14-4c0e-8c52-c207245348a4-kube-api-access-nrk4f\") pod \"test-curl-7284-140051076097632\" (UID: \"43166304-9d14-4c0e-8c52-c207245348a4\") " pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:06.528405 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.528378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:06.654118 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.654073    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:06.656374 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:11:06.656349    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43166304_9d14_4c0e_8c52_c207245348a4.slice/crio-aab9f3d41c208bb0d6964136967420b026c278504e6b196603fa8bef6be5329e WatchSource:0}: Error finding container aab9f3d41c208bb0d6964136967420b026c278504e6b196603fa8bef6be5329e: Status 404 returned error can't find the container with id aab9f3d41c208bb0d6964136967420b026c278504e6b196603fa8bef6be5329e
Jul 27 18:11:06.913877 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.913798    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1b0cf33ffc321bc856d009cc8d093444bbe446841bd65f6772dd749b5b150cb"
Jul 27 18:11:06.913877 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.913804    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:06.915052 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.915027    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076097632" event={"ID":"43166304-9d14-4c0e-8c52-c207245348a4","Type":"ContainerStarted","Data":"aab9f3d41c208bb0d6964136967420b026c278504e6b196603fa8bef6be5329e"}
Jul 27 18:11:06.926117 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:06.926094    2565 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7284-140051076097632" oldPodUID="3e29f6b1-3156-460a-9e31-5aa55c283963" podUID="43166304-9d14-4c0e-8c52-c207245348a4"
Jul 27 18:11:07.255952 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:07.255874    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8e9814-3cfa-4070-9e9b-09f00b419722" path="/var/lib/kubelet/pods/1a8e9814-3cfa-4070-9e9b-09f00b419722/volumes"
Jul 27 18:11:07.256339 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:07.256321    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e676278-5ccb-4ff0-9dcb-25a8b86627f1" path="/var/lib/kubelet/pods/2e676278-5ccb-4ff0-9dcb-25a8b86627f1/volumes"
Jul 27 18:11:07.256661 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:07.256646    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e29f6b1-3156-460a-9e31-5aa55c283963" path="/var/lib/kubelet/pods/3e29f6b1-3156-460a-9e31-5aa55c283963/volumes"
Jul 27 18:11:07.920873 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:07.920839    2565 generic.go:356] "Generic (PLEG): container finished" podID="43166304-9d14-4c0e-8c52-c207245348a4" containerID="563ea21e3ce782d6f66f1cca917d0a3440d6328ec2780901ddd32b8021d93780" exitCode=0
Jul 27 18:11:07.921229 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:07.920917    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7284-140051076097632" event={"ID":"43166304-9d14-4c0e-8c52-c207245348a4","Type":"ContainerDied","Data":"563ea21e3ce782d6f66f1cca917d0a3440d6328ec2780901ddd32b8021d93780"}
Jul 27 18:11:09.073796 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.073772    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:09.094453 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.094431    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7284-140051076097632_43166304-9d14-4c0e-8c52-c207245348a4/test-curl-7284-140051076097632/0.log"
Jul 27 18:11:09.117058 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.117033    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:09.120550 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.120528    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7284-140051076097632"]
Jul 27 18:11:09.219320 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.219233    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrk4f\" (UniqueName: \"kubernetes.io/projected/43166304-9d14-4c0e-8c52-c207245348a4-kube-api-access-nrk4f\") pod \"43166304-9d14-4c0e-8c52-c207245348a4\" (UID: \"43166304-9d14-4c0e-8c52-c207245348a4\") "
Jul 27 18:11:09.221356 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.221336    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43166304-9d14-4c0e-8c52-c207245348a4-kube-api-access-nrk4f" (OuterVolumeSpecName: "kube-api-access-nrk4f") pod "43166304-9d14-4c0e-8c52-c207245348a4" (UID: "43166304-9d14-4c0e-8c52-c207245348a4"). InnerVolumeSpecName "kube-api-access-nrk4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:11:09.255880 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.255852    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43166304-9d14-4c0e-8c52-c207245348a4" path="/var/lib/kubelet/pods/43166304-9d14-4c0e-8c52-c207245348a4/volumes"
Jul 27 18:11:09.320848 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.320823    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrk4f\" (UniqueName: \"kubernetes.io/projected/43166304-9d14-4c0e-8c52-c207245348a4-kube-api-access-nrk4f\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:11:09.936235 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.936181    2565 scope.go:117] "RemoveContainer" containerID="563ea21e3ce782d6f66f1cca917d0a3440d6328ec2780901ddd32b8021d93780"
Jul 27 18:11:09.936235 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:09.936204    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7284-140051076097632"
Jul 27 18:11:18.886948 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:18.886910    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:11:41.670044 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:11:41.669964    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:12:08.136652 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:08.136598    2565 controller.go:195] "Failed to update lease" err="Put \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-8.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 27 18:12:18.137082 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:18.137029    2565 controller.go:195] "Failed to update lease" err="Put \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-8.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 27 18:12:26.851268 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:26.851210    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T18:12:16Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T18:12:16Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T18:12:16Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-27T18:12:16Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-131-8.ec2.internal\": Patch \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-131-8.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 27 18:12:28.137496 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:28.137453    2565 controller.go:195] "Failed to update lease" err="Put \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-8.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 27 18:12:36.851952 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:36.851900    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-8.ec2.internal\": Get \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-131-8.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 27 18:12:38.138239 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:38.138197    2565 controller.go:195] "Failed to update lease" err="Put \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-8.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 27 18:12:42.682823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682775    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.682823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682806    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.682823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682822    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682854    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qg782\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682860    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682883    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gv7q2\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682889    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682892    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682923    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682929    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682950    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:42.682960    2565 controller.go:195] "Failed to update lease" err="Put \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-131-8.ec2.internal?timeout=10s\": http2: client connection lost"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.682985    2565 controller.go:115] "failed to update lease using latest lease, fallback to ensure lease" err="failed 5 attempts to update lease"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683000    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683026    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683051    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-k55f8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683053    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683070    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dgczs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683077    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683079    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683117    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.683429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683138    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nqddh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683168    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683191    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pjdbj\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683207    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683212    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tgcgh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683224    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683234    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pxtmn\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683260    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xp4lh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683266    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683279    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wnjmr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683319    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683322    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-dockercfg-wpb8l\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683348    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-np22g\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683354    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683361    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683374    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683401    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683412    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684386 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683410    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683422    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8fm7v\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683436    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683452    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683484    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683484    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683503    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"default-dockercfg-scg5l\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683520    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qr2nv\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683555    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683562    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683565    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683576    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683595    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683599    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683608    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683615    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683639    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hgdfc\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683640    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.684998 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683659    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683685    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2k848\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683694    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k48sk\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683702    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bfsxf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683708    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683736    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683740    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683754    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683767    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683802    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-428tm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683806    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4fhbz\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683814    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683861    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w7cjs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683869    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683876    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c6k2r\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683883    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683908    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683911    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.685567 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683923    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683967    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6chq\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683971    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"default-dockercfg-zlcs9\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683984    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683990    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6dfng\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683994    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684011    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"default-dockercfg-9j5fv\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684011    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683987    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684015    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683993    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684034    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.683991    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684054    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684054    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684073    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684078    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684073    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686116 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684104    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684104    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bgcdf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684101    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684118    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684121    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684114    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684134    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684139    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684143    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684157    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684169    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684194    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684200    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684204    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684215    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684220    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684226    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684228    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.686675 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684244    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684245    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684254    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684258    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684261    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684261    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684271    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xqfrl\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684279    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684285    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pc4tx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684288    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684313    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684320    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684324    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684326    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vl4wk\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684344    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bvxqm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684347    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684358    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684364    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687249 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684369    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684374    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kjpq5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684381    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684394    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684397    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684394    2565 reflector.go:571] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684394    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684406    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684425    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684430    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684434    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684410    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfvk\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684455    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684465    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684470    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rf7vh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684485    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684488    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684493    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.687823 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684500    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.688367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684509    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.688367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:42.684510    2565 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Jul 27 18:12:42.688367 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:42.684626    2565 token_manager.go:124] "Couldn't update token" err="Post \"https://a9625b2ca139d492684213f5799dca2a-c689789764f3c9f1.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-monitoring/serviceaccounts/telemeter-client/token\": http2: client connection lost" cacheKey="\"telemeter-client\"/\"openshift-monitoring\"/[]string(nil)/3607/v1.BoundObjectReference{Kind:\"Pod\", APIVersion:\"v1\", Name:\"telemeter-client-657c6c8b9d-5lgw7\", UID:\"3da258f4-720e-4bc8-992f-2e73584c4948\"}/\"\""
Jul 27 18:12:43.489206 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.489169    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.491223 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.491201    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.498505 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.498488    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.499304 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.499276    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.504783 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.504769    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 18:12:43.505995 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.505980    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zlcs9\""
Jul 27 18:12:43.509085 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.509069    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 27 18:12:43.510829 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.510815    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.511749 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.511732    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 27 18:12:43.524422 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.524406    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.526967 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.526952    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 18:12:43.529634 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.529619    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 18:12:43.530148 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.530134    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 18:12:43.532732 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.532719    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-k55f8\""
Jul 27 18:12:43.565415 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.565397    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 18:12:43.565501 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.565453    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.582279 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.582260    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.582347 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.582327    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rf7vh\""
Jul 27 18:12:43.596438 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.596421    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bfsxf\""
Jul 27 18:12:43.597274 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.597260    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 27 18:12:43.602765 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.602749    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.604380 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.604368    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.606303 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.606278    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k48sk\""
Jul 27 18:12:43.618591 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.618569    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bvxqm\""
Jul 27 18:12:43.619586 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.619571    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 18:12:43.622926 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.622911    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w7cjs\""
Jul 27 18:12:43.625591 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.625575    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hgdfc\""
Jul 27 18:12:43.627255 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.627238    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 18:12:43.627349 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.627267    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c6k2r\""
Jul 27 18:12:43.628456 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.628442    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bgcdf\""
Jul 27 18:12:43.630508 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.630492    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:12:43.632604 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.632590    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 18:12:43.640662 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.640650    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.651177 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.646323    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2k848\""
Jul 27 18:12:43.659556 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.659532    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 18:12:43.667040 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.667020    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 18:12:43.673346 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.673332    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.688903 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.688880    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 18:12:43.689253 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.688916    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 18:12:43.689253 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.688930    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeHasSufficientPID"
Jul 27 18:12:43.689253 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.688947    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-8.ec2.internal" event="NodeReady"
Jul 27 18:12:43.698174 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.698156    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 18:12:43.704387 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.704373    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.706403 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.706387    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.706476 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.706402    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 18:12:43.713731 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.713716    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 18:12:43.730876 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.730855    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.740393 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.740340    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.745914 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.745897    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 18:12:43.751534 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.751517    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dgczs\""
Jul 27 18:12:43.754090 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.754074    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:12:43.770087 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.770065    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:12:43.791350 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.791327    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 27 18:12:43.791435 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.791395    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8fm7v\""
Jul 27 18:12:43.792898 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.792883    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kjpq5\""
Jul 27 18:12:43.796812 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.796796    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qg782\""
Jul 27 18:12:43.799272 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.799257    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pxtmn\""
Jul 27 18:12:43.807436 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.807422    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wpb8l\""
Jul 27 18:12:43.809801 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.809787    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qr2nv\""
Jul 27 18:12:43.809866 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.809853    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.823946 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.823927    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.824448 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.824427    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.824542 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.824465    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tgcgh\""
Jul 27 18:12:43.827415 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.827400    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.840833 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.840813    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6chq\""
Jul 27 18:12:43.846084 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.846068    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 18:12:43.861659 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.861636    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.881953 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.881933    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.884473 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.884458    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 18:12:43.887960 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.887943    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9j5fv\""
Jul 27 18:12:43.891827 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.891812    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 27 18:12:43.904050 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.904029    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 18:12:43.910598 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.910579    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 27 18:12:43.920469 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.920450    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gv7q2\""
Jul 27 18:12:43.923808 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.923791    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 27 18:12:43.925540 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.925526    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.925622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.925527    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.930329 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.930311    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.934558 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.934541    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pjdbj\""
Jul 27 18:12:43.939989 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.939973    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 18:12:43.943304 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.943279    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 27 18:12:43.949797 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.949777    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nqddh\""
Jul 27 18:12:43.951156 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.951142    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 18:12:43.963543 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.963526    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.967179 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.967165    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 18:12:43.972180 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.972164    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 18:12:43.974330 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.974311    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.978660 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.978643    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.981746 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.981730    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 18:12:43.986758 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.986744    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 18:12:43.998150 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.998097    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 18:12:43.998454 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.998441    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vl4wk\""
Jul 27 18:12:43.998504 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:43.998458    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xqfrl\""
Jul 27 18:12:44.007119 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.007099    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.009684 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.009667    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 27 18:12:44.018071 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.018056    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.020843 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.020824    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.021433 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.021417    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-scg5l\""
Jul 27 18:12:44.021749 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.021733    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pc4tx\""
Jul 27 18:12:44.024246 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.024223    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.024384 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.024368    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 18:12:44.027863 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.027847    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 18:12:44.030314 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.030272    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 18:12:44.035839 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.035824    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.044429 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.044411    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.048069 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.048054    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 18:12:44.050091 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.050074    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 18:12:44.051908 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.051892    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.051961 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.051892    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.052560 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.052547    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 18:12:44.052661 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.052645    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xp4lh\""
Jul 27 18:12:44.072990 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.072970    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 18:12:44.076703 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.076688    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 18:12:44.081253 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.081240    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 18:12:44.091427 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.091411    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 18:12:44.094503 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.094489    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 18:12:44.106495 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.106480    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 18:12:44.113992 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.113977    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.126805 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.126783    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.126805 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.126791    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 18:12:44.130496 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.130478    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-428tm\""
Jul 27 18:12:44.135186 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.135171    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.136903 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.136887    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 18:12:44.140212 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.140197    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 18:12:44.141841 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.141828    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 18:12:44.153442 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.153423    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.154021 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.154008    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 18:12:44.159191 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.159175    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.172555 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.172535    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 27 18:12:44.189560 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.189538    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 27 18:12:44.189662 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.189539    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.199830 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.199812    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 18:12:44.202548 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.202532    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 27 18:12:44.210417 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.210402    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 18:12:44.216994 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.216977    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.228731 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.228710    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 18:12:44.235539 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.235520    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.237610 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.237594    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 18:12:44.237817 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.237800    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 18:12:44.240585 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.240571    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4fhbz\""
Jul 27 18:12:44.245154 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.245139    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fmfvk\""
Jul 27 18:12:44.249594 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.249546    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 27 18:12:44.270810 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.270791    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6dfng\""
Jul 27 18:12:44.289392 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.289375    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 18:12:44.321481 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.321461    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 18:12:44.323777 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.323760    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"]
Jul 27 18:12:44.324072 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.324061    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43166304-9d14-4c0e-8c52-c207245348a4" containerName="test-curl-7284-140051076097632"
Jul 27 18:12:44.324109 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.324075    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="43166304-9d14-4c0e-8c52-c207245348a4" containerName="test-curl-7284-140051076097632"
Jul 27 18:12:44.324155 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.324147    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="43166304-9d14-4c0e-8c52-c207245348a4" containerName="test-curl-7284-140051076097632"
Jul 27 18:12:44.326930 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.326911    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:12:44.327057 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.327043    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.335313 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.335274    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 27 18:12:44.352207 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.352190    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wnjmr\""
Jul 27 18:12:44.369887 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.369867    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 18:12:44.389634 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.389613    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 18:12:44.395759 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.395737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tls-certs\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.395837 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.395784    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-model-cache\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.395837 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.395805    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-home\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.395944 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.395932    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tmp-dir\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.395989 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.395973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-dshm\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.396025 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.395988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkwkq\" (UniqueName: \"kubernetes.io/projected/3fb6d227-df54-4854-a7b9-4eb93a79d56c-kube-api-access-kkwkq\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.410406 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.410390    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.429442 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.429423    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.449285 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.449263    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-np22g\""
Jul 27 18:12:44.469700 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.469680    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7985e5\"/\"default-dockercfg-9x4tt\""
Jul 27 18:12:44.491566 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.491543    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7985e5\"/\"ipp-route-7985e5-kserve-self-signed-certs\""
Jul 27 18:12:44.496679 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.496658    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tmp-dir\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.496758 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.496701    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-dshm\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.496758 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.496721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkwkq\" (UniqueName: \"kubernetes.io/projected/3fb6d227-df54-4854-a7b9-4eb93a79d56c-kube-api-access-kkwkq\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.496758 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.496755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tls-certs\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.496920 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.496784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-model-cache\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.496920 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.496800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-home\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.497099 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.497069    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tmp-dir\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.497205 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.497172    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-home\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.497261 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.497193    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-model-cache\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.499018 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.498984    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-dshm\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.499107 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.499090    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tls-certs\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.509606 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.509545    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7985e5\"/\"kube-root-ca.crt\""
Jul 27 18:12:44.549611 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.549593    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7985e5\"/\"openshift-service-ca.crt\""
Jul 27 18:12:44.578725 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.578698    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkwkq\" (UniqueName: \"kubernetes.io/projected/3fb6d227-df54-4854-a7b9-4eb93a79d56c-kube-api-access-kkwkq\") pod \"ipp-route-7985e5-kserve-8679856bcf-4bpgt\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") " pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.636625 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.636598    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:44.763710 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:44.763686    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"]
Jul 27 18:12:44.766101 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:12:44.766074    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fb6d227_df54_4854_a7b9_4eb93a79d56c.slice/crio-590ef5050d285c72bc3f51914ab343bc84fab4305040d834b3f4d01cf54d54d8 WatchSource:0}: Error finding container 590ef5050d285c72bc3f51914ab343bc84fab4305040d834b3f4d01cf54d54d8: Status 404 returned error can't find the container with id 590ef5050d285c72bc3f51914ab343bc84fab4305040d834b3f4d01cf54d54d8
Jul 27 18:12:45.052905 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.052872    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tm2l"]
Jul 27 18:12:45.055965 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.055948    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:45.058150 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.058127    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2z8xz\""
Jul 27 18:12:45.070755 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.070735    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tm2l"]
Jul 27 18:12:45.103096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.103075    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vws76\" (UniqueName: \"kubernetes.io/projected/2ce243b5-c552-4cd6-b72a-e56ea2d16cfa-kube-api-access-vws76\") pod \"kuadrant-operator-catalog-9tm2l\" (UID: \"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa\") " pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:45.204085 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.204058    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vws76\" (UniqueName: \"kubernetes.io/projected/2ce243b5-c552-4cd6-b72a-e56ea2d16cfa-kube-api-access-vws76\") pod \"kuadrant-operator-catalog-9tm2l\" (UID: \"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa\") " pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:45.224899 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.224879    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vws76\" (UniqueName: \"kubernetes.io/projected/2ce243b5-c552-4cd6-b72a-e56ea2d16cfa-kube-api-access-vws76\") pod \"kuadrant-operator-catalog-9tm2l\" (UID: \"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa\") " pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:45.365908 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.365818    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:45.375622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.375590    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt" event={"ID":"3fb6d227-df54-4854-a7b9-4eb93a79d56c","Type":"ContainerStarted","Data":"0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d"}
Jul 27 18:12:45.375756 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.375634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt" event={"ID":"3fb6d227-df54-4854-a7b9-4eb93a79d56c","Type":"ContainerStarted","Data":"590ef5050d285c72bc3f51914ab343bc84fab4305040d834b3f4d01cf54d54d8"}
Jul 27 18:12:45.414829 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.414759    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt" podStartSLOduration=25.414726754 podStartE2EDuration="25.414726754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:12:45.41169179 +0000 UTC m=+3122.672666990" watchObservedRunningTime="2026-07-27 18:12:45.414726754 +0000 UTC m=+3122.675701957"
Jul 27 18:12:45.501140 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:45.501109    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tm2l"]
Jul 27 18:12:45.502738 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:12:45.502698    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce243b5_c552_4cd6_b72a_e56ea2d16cfa.slice/crio-af3fd52fca3cacebea0afda0337cd18a4436c782b4a1d35f0f0396a7bc1f7af7 WatchSource:0}: Error finding container af3fd52fca3cacebea0afda0337cd18a4436c782b4a1d35f0f0396a7bc1f7af7: Status 404 returned error can't find the container with id af3fd52fca3cacebea0afda0337cd18a4436c782b4a1d35f0f0396a7bc1f7af7
Jul 27 18:12:46.382274 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:46.382239    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l" event={"ID":"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa","Type":"ContainerStarted","Data":"af3fd52fca3cacebea0afda0337cd18a4436c782b4a1d35f0f0396a7bc1f7af7"}
Jul 27 18:12:48.394396 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:48.394353    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l" event={"ID":"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa","Type":"ContainerStarted","Data":"882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0"}
Jul 27 18:12:48.423975 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:48.423926    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l" podStartSLOduration=1.189215608 podStartE2EDuration="3.423911351s" totalImagesPullingTime="2.234695743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:12:48.422642021 +0000 UTC m=+3125.683617220" watchObservedRunningTime="2026-07-27 18:12:48.423911351 +0000 UTC m=+3125.684886550"
Jul 27 18:12:54.637620 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:54.637583    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:54.638054 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:54.637632    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:54.650307 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:54.650273    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:55.366869 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:55.366828    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:55.367091 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:55.366890    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:55.389179 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:55.389151    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:55.440719 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:55.440688    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:12:55.450463 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:55.450440    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:55.619843 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:55.619752    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tm2l"]
Jul 27 18:12:57.438861 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:57.438800    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l" podUID="2ce243b5-c552-4cd6-b72a-e56ea2d16cfa" containerName="registry-server" containerID="cri-o://882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0" gracePeriod=2
Jul 27 18:12:57.680721 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:57.680699    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:57.807614 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:57.807581    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vws76\" (UniqueName: \"kubernetes.io/projected/2ce243b5-c552-4cd6-b72a-e56ea2d16cfa-kube-api-access-vws76\") pod \"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa\" (UID: \"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa\") "
Jul 27 18:12:57.809622 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:57.809600    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce243b5-c552-4cd6-b72a-e56ea2d16cfa-kube-api-access-vws76" (OuterVolumeSpecName: "kube-api-access-vws76") pod "2ce243b5-c552-4cd6-b72a-e56ea2d16cfa" (UID: "2ce243b5-c552-4cd6-b72a-e56ea2d16cfa"). InnerVolumeSpecName "kube-api-access-vws76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:12:57.908218 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:57.908183    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vws76\" (UniqueName: \"kubernetes.io/projected/2ce243b5-c552-4cd6-b72a-e56ea2d16cfa-kube-api-access-vws76\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:12:58.444465 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.444429    2565 generic.go:356] "Generic (PLEG): container finished" podID="2ce243b5-c552-4cd6-b72a-e56ea2d16cfa" containerID="882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0" exitCode=0
Jul 27 18:12:58.444877 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.444484    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l"
Jul 27 18:12:58.444877 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.444491    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l" event={"ID":"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa","Type":"ContainerDied","Data":"882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0"}
Jul 27 18:12:58.444877 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.444530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9tm2l" event={"ID":"2ce243b5-c552-4cd6-b72a-e56ea2d16cfa","Type":"ContainerDied","Data":"af3fd52fca3cacebea0afda0337cd18a4436c782b4a1d35f0f0396a7bc1f7af7"}
Jul 27 18:12:58.444877 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.444549    2565 scope.go:117] "RemoveContainer" containerID="882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0"
Jul 27 18:12:58.453596 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.453581    2565 scope.go:117] "RemoveContainer" containerID="882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0"
Jul 27 18:12:58.453878 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:12:58.453861    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0\": container with ID starting with 882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0 not found: ID does not exist" containerID="882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0"
Jul 27 18:12:58.453930 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.453887    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0"} err="failed to get container status \"882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0\": rpc error: code = NotFound desc = could not find container \"882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0\": container with ID starting with 882e2d209aff4237e73d492211692bcf1df31f7993ff986f75603ec2e9f3e4a0 not found: ID does not exist"
Jul 27 18:12:58.464903 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.464879    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tm2l"]
Jul 27 18:12:58.468885 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:58.468862    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9tm2l"]
Jul 27 18:12:59.255932 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:12:59.255899    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ce243b5-c552-4cd6-b72a-e56ea2d16cfa" path="/var/lib/kubelet/pods/2ce243b5-c552-4cd6-b72a-e56ea2d16cfa/volumes"
Jul 27 18:13:16.984636 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:16.984601    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:13:43.617607 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:43.617578    2565 scope.go:117] "RemoveContainer" containerID="7d0db53432c769ca416d408c64015c78b576884b3969696d4bb89fd5b8875519"
Jul 27 18:13:44.082532 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:44.082499    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:13:53.343452 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.343415    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"]
Jul 27 18:13:53.344048 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.343906    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ce243b5-c552-4cd6-b72a-e56ea2d16cfa" containerName="registry-server"
Jul 27 18:13:53.344048 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.343926    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ce243b5-c552-4cd6-b72a-e56ea2d16cfa" containerName="registry-server"
Jul 27 18:13:53.344048 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.344020    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ce243b5-c552-4cd6-b72a-e56ea2d16cfa" containerName="registry-server"
Jul 27 18:13:53.348472 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.348448    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"
Jul 27 18:13:53.350701 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.350682    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:13:53.350820 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.350716    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 18:13:53.357685 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.357660    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"]
Jul 27 18:13:53.454834 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.454806    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a82bcaa-a1ec-4a57-958f-bc5cd9248d13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl\" (UID: \"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"
Jul 27 18:13:53.555626 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.555598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a82bcaa-a1ec-4a57-958f-bc5cd9248d13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl\" (UID: \"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"
Jul 27 18:13:53.556157 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.556140    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a82bcaa-a1ec-4a57-958f-bc5cd9248d13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl\" (UID: \"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"
Jul 27 18:13:53.659556 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.659491    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"
Jul 27 18:13:53.785922 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:53.785864    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"]
Jul 27 18:13:53.788241 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:13:53.788212    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a82bcaa_a1ec_4a57_958f_bc5cd9248d13.slice/crio-89a9c716db3a0f6319439167e154cb70a0b07afde229c7ab53914c4d484417e5 WatchSource:0}: Error finding container 89a9c716db3a0f6319439167e154cb70a0b07afde229c7ab53914c4d484417e5: Status 404 returned error can't find the container with id 89a9c716db3a0f6319439167e154cb70a0b07afde229c7ab53914c4d484417e5
Jul 27 18:13:54.712616 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:54.712524    2565 generic.go:356] "Generic (PLEG): container finished" podID="0a82bcaa-a1ec-4a57-958f-bc5cd9248d13" containerID="9e1434766031bf9d5ab3f3ca3822a50ce6bcd1901d350cb435ecd5b09265376f" exitCode=0
Jul 27 18:13:54.712616 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:54.712565    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl" event={"ID":"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13","Type":"ContainerDied","Data":"9e1434766031bf9d5ab3f3ca3822a50ce6bcd1901d350cb435ecd5b09265376f"}
Jul 27 18:13:54.712616 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:54.712602    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl" event={"ID":"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13","Type":"ContainerStarted","Data":"89a9c716db3a0f6319439167e154cb70a0b07afde229c7ab53914c4d484417e5"}
Jul 27 18:13:55.846214 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:55.846190    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"
Jul 27 18:13:55.874245 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:55.874218    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a82bcaa-a1ec-4a57-958f-bc5cd9248d13-maas-api-service-ca\") pod \"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13\" (UID: \"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13\") "
Jul 27 18:13:55.874572 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:55.874550    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a82bcaa-a1ec-4a57-958f-bc5cd9248d13-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a82bcaa-a1ec-4a57-958f-bc5cd9248d13" (UID: "0a82bcaa-a1ec-4a57-958f-bc5cd9248d13"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:13:55.975817 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:55.975723    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a82bcaa-a1ec-4a57-958f-bc5cd9248d13-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:13:56.723364 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:56.723330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl" event={"ID":"0a82bcaa-a1ec-4a57-958f-bc5cd9248d13","Type":"ContainerDied","Data":"89a9c716db3a0f6319439167e154cb70a0b07afde229c7ab53914c4d484417e5"}
Jul 27 18:13:56.723364 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:56.723364    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"
Jul 27 18:13:56.723588 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:13:56.723418    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89a9c716db3a0f6319439167e154cb70a0b07afde229c7ab53914c4d484417e5"
Jul 27 18:14:08.418477 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:08.418446    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"]
Jul 27 18:14:08.422955 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:08.422931    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-e962ab-s8qbl"]
Jul 27 18:14:09.262587 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:09.262555    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a82bcaa-a1ec-4a57-958f-bc5cd9248d13" path="/var/lib/kubelet/pods/0a82bcaa-a1ec-4a57-958f-bc5cd9248d13/volumes"
Jul 27 18:14:10.872090 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:10.872054    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:14:18.609191 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.609155    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"]
Jul 27 18:14:18.609618 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.609537    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a82bcaa-a1ec-4a57-958f-bc5cd9248d13" containerName="revoke-keys"
Jul 27 18:14:18.609618 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.609550    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a82bcaa-a1ec-4a57-958f-bc5cd9248d13" containerName="revoke-keys"
Jul 27 18:14:18.609618 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.609613    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a82bcaa-a1ec-4a57-958f-bc5cd9248d13" containerName="revoke-keys"
Jul 27 18:14:18.613691 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.613673    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"
Jul 27 18:14:18.615773 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.615736    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 18:14:18.615888 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.615818    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:14:18.621026 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.620934    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"]
Jul 27 18:14:18.771361 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.771328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/787de9b6-a6d5-4289-91ca-2d0ce60ed4ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp\" (UID: \"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"
Jul 27 18:14:18.872564 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.872480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/787de9b6-a6d5-4289-91ca-2d0ce60ed4ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp\" (UID: \"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"
Jul 27 18:14:18.873126 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.873106    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/787de9b6-a6d5-4289-91ca-2d0ce60ed4ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp\" (UID: \"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"
Jul 27 18:14:18.924564 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:18.924541    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"
Jul 27 18:14:19.055499 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:19.055471    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"]
Jul 27 18:14:19.058160 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:14:19.058137    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod787de9b6_a6d5_4289_91ca_2d0ce60ed4ae.slice/crio-90f21fe6aca64ea96048e327dea9a416893600d627b856088e7549767536b96c WatchSource:0}: Error finding container 90f21fe6aca64ea96048e327dea9a416893600d627b856088e7549767536b96c: Status 404 returned error can't find the container with id 90f21fe6aca64ea96048e327dea9a416893600d627b856088e7549767536b96c
Jul 27 18:14:19.831364 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:19.831327    2565 generic.go:356] "Generic (PLEG): container finished" podID="787de9b6-a6d5-4289-91ca-2d0ce60ed4ae" containerID="0e02d2242ab699a2bef70c330b2bbbe349aaf7bf199cf23fba6d044fd552725d" exitCode=0
Jul 27 18:14:19.831763 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:19.831363    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp" event={"ID":"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae","Type":"ContainerDied","Data":"0e02d2242ab699a2bef70c330b2bbbe349aaf7bf199cf23fba6d044fd552725d"}
Jul 27 18:14:19.831763 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:19.831415    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp" event={"ID":"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae","Type":"ContainerStarted","Data":"90f21fe6aca64ea96048e327dea9a416893600d627b856088e7549767536b96c"}
Jul 27 18:14:20.964160 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:20.964137    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"
Jul 27 18:14:21.092739 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:21.092661    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/787de9b6-a6d5-4289-91ca-2d0ce60ed4ae-maas-api-service-ca\") pod \"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae\" (UID: \"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae\") "
Jul 27 18:14:21.092994 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:21.092972    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/787de9b6-a6d5-4289-91ca-2d0ce60ed4ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "787de9b6-a6d5-4289-91ca-2d0ce60ed4ae" (UID: "787de9b6-a6d5-4289-91ca-2d0ce60ed4ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:14:21.194041 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:21.194011    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/787de9b6-a6d5-4289-91ca-2d0ce60ed4ae-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:14:21.842759 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:21.842726    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"
Jul 27 18:14:21.842935 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:21.842762    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp" event={"ID":"787de9b6-a6d5-4289-91ca-2d0ce60ed4ae","Type":"ContainerDied","Data":"90f21fe6aca64ea96048e327dea9a416893600d627b856088e7549767536b96c"}
Jul 27 18:14:21.842935 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:21.842813    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90f21fe6aca64ea96048e327dea9a416893600d627b856088e7549767536b96c"
Jul 27 18:14:35.878608 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:35.878523    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:14:38.697016 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:38.696983    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"]
Jul 27 18:14:38.701138 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:38.701114    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7985e5-4ppkp"]
Jul 27 18:14:39.255683 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:39.255649    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="787de9b6-a6d5-4289-91ca-2d0ce60ed4ae" path="/var/lib/kubelet/pods/787de9b6-a6d5-4289-91ca-2d0ce60ed4ae/volumes"
Jul 27 18:14:44.039089 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:44.039053    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"]
Jul 27 18:14:47.197094 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:47.197059    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:14:59.057801 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.057747    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt" podUID="3fb6d227-df54-4854-a7b9-4eb93a79d56c" containerName="main" containerID="cri-o://0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d" gracePeriod=45
Jul 27 18:14:59.312929 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.312872    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:14:59.388723 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388684    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-home\") pod \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") "
Jul 27 18:14:59.388723 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388716    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkwkq\" (UniqueName: \"kubernetes.io/projected/3fb6d227-df54-4854-a7b9-4eb93a79d56c-kube-api-access-kkwkq\") pod \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") "
Jul 27 18:14:59.388974 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388754    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tmp-dir\") pod \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") "
Jul 27 18:14:59.388974 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388846    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-model-cache\") pod \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") "
Jul 27 18:14:59.388974 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388899    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tls-certs\") pod \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") "
Jul 27 18:14:59.388974 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388926    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-dshm\") pod \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\" (UID: \"3fb6d227-df54-4854-a7b9-4eb93a79d56c\") "
Jul 27 18:14:59.389152 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388991    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-home" (OuterVolumeSpecName: "home") pod "3fb6d227-df54-4854-a7b9-4eb93a79d56c" (UID: "3fb6d227-df54-4854-a7b9-4eb93a79d56c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:59.389152 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.388996    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3fb6d227-df54-4854-a7b9-4eb93a79d56c" (UID: "3fb6d227-df54-4854-a7b9-4eb93a79d56c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:59.389152 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.389104    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-model-cache" (OuterVolumeSpecName: "model-cache") pod "3fb6d227-df54-4854-a7b9-4eb93a79d56c" (UID: "3fb6d227-df54-4854-a7b9-4eb93a79d56c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:59.389261 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.389200    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-home\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:14:59.389261 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.389218    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tmp-dir\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:14:59.389261 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.389233    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-model-cache\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:14:59.390898 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.390868    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb6d227-df54-4854-a7b9-4eb93a79d56c-kube-api-access-kkwkq" (OuterVolumeSpecName: "kube-api-access-kkwkq") pod "3fb6d227-df54-4854-a7b9-4eb93a79d56c" (UID: "3fb6d227-df54-4854-a7b9-4eb93a79d56c"). InnerVolumeSpecName "kube-api-access-kkwkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:14:59.391025 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.390900    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-dshm" (OuterVolumeSpecName: "dshm") pod "3fb6d227-df54-4854-a7b9-4eb93a79d56c" (UID: "3fb6d227-df54-4854-a7b9-4eb93a79d56c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:14:59.391025 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.390970    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3fb6d227-df54-4854-a7b9-4eb93a79d56c" (UID: "3fb6d227-df54-4854-a7b9-4eb93a79d56c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:14:59.490583 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.490549    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb6d227-df54-4854-a7b9-4eb93a79d56c-tls-certs\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:14:59.490583 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.490578    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb6d227-df54-4854-a7b9-4eb93a79d56c-dshm\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:14:59.490583 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:14:59.490588    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkwkq\" (UniqueName: \"kubernetes.io/projected/3fb6d227-df54-4854-a7b9-4eb93a79d56c-kube-api-access-kkwkq\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:15:00.022930 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.022890    2565 generic.go:356] "Generic (PLEG): container finished" podID="3fb6d227-df54-4854-a7b9-4eb93a79d56c" containerID="0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d" exitCode=0
Jul 27 18:15:00.023159 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.022952    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"
Jul 27 18:15:00.023159 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.022999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt" event={"ID":"3fb6d227-df54-4854-a7b9-4eb93a79d56c","Type":"ContainerDied","Data":"0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d"}
Jul 27 18:15:00.023159 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.023054    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt" event={"ID":"3fb6d227-df54-4854-a7b9-4eb93a79d56c","Type":"ContainerDied","Data":"590ef5050d285c72bc3f51914ab343bc84fab4305040d834b3f4d01cf54d54d8"}
Jul 27 18:15:00.023159 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.023078    2565 scope.go:117] "RemoveContainer" containerID="0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d"
Jul 27 18:15:00.032530 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.032516    2565 scope.go:117] "RemoveContainer" containerID="0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d"
Jul 27 18:15:00.032770 ip-10-0-131-8 kubenswrapper[2565]: E0727 18:15:00.032752    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d\": container with ID starting with 0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d not found: ID does not exist" containerID="0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d"
Jul 27 18:15:00.032831 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.032776    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d"} err="failed to get container status \"0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d\": rpc error: code = NotFound desc = could not find container \"0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d\": container with ID starting with 0cdfe121c2e3e0e95353828968497b81cfc175b21e1abf9265ea1e7521cf9c4d not found: ID does not exist"
Jul 27 18:15:00.051846 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.051824    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"]
Jul 27 18:15:00.055783 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.055764    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-7985e5/ipp-route-7985e5-kserve-8679856bcf-4bpgt"]
Jul 27 18:15:00.144853 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.144824    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"]
Jul 27 18:15:00.145195 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.145180    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fb6d227-df54-4854-a7b9-4eb93a79d56c" containerName="main"
Jul 27 18:15:00.145243 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.145196    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fb6d227-df54-4854-a7b9-4eb93a79d56c" containerName="main"
Jul 27 18:15:00.145243 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.145218    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="787de9b6-a6d5-4289-91ca-2d0ce60ed4ae" containerName="revoke-keys"
Jul 27 18:15:00.145243 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.145224    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="787de9b6-a6d5-4289-91ca-2d0ce60ed4ae" containerName="revoke-keys"
Jul 27 18:15:00.145371 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.145280    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fb6d227-df54-4854-a7b9-4eb93a79d56c" containerName="main"
Jul 27 18:15:00.145371 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.145306    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="787de9b6-a6d5-4289-91ca-2d0ce60ed4ae" containerName="revoke-keys"
Jul 27 18:15:00.149448 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.149426    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b"]
Jul 27 18:15:00.149566 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.149551    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"
Jul 27 18:15:00.151553 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.151536    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:15:00.152596 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.152581    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"]
Jul 27 18:15:00.152690 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.152678    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b"
Jul 27 18:15:00.155581 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.155564    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"]
Jul 27 18:15:00.155653 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.155619    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"
Jul 27 18:15:00.159074 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.158859    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b"]
Jul 27 18:15:00.159074 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.158958    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"
Jul 27 18:15:00.160697 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.160661    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"]
Jul 27 18:15:00.162262 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.162243    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"
Jul 27 18:15:00.162505 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.162484    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"]
Jul 27 18:15:00.172257 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.172240    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b"
Jul 27 18:15:00.176320 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.176255    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"]
Jul 27 18:15:00.207917 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.207886    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"
Jul 27 18:15:00.215734 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.215712    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"
Jul 27 18:15:00.353852 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.353821    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"]
Jul 27 18:15:00.356388 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:15:00.356317    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa897af7_a766_4d16_a951_860227c924be.slice/crio-8769b1b62eb803aacc294f4b6da2b8fde041159c55bebe697f1a6b9cbe071896 WatchSource:0}: Error finding container 8769b1b62eb803aacc294f4b6da2b8fde041159c55bebe697f1a6b9cbe071896: Status 404 returned error can't find the container with id 8769b1b62eb803aacc294f4b6da2b8fde041159c55bebe697f1a6b9cbe071896
Jul 27 18:15:00.587168 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.587135    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b"]
Jul 27 18:15:00.590107 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:15:00.590080    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf131ae91_f29b_4ff9_97b0_06cb3c1ddb99.slice/crio-8a2c68288dbc0f07d327c3f491cdf19ac59359e591061ed6986ac19e8ed2f342 WatchSource:0}: Error finding container 8a2c68288dbc0f07d327c3f491cdf19ac59359e591061ed6986ac19e8ed2f342: Status 404 returned error can't find the container with id 8a2c68288dbc0f07d327c3f491cdf19ac59359e591061ed6986ac19e8ed2f342
Jul 27 18:15:00.620796 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.620771    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"]
Jul 27 18:15:00.623169 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:00.623149    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"]
Jul 27 18:15:00.624544 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:15:00.624519    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512d18a1_e88a_4a2c_b2d9_7b86647987b0.slice/crio-4819ed6e5da372be0f670059d3ee9c1d41dd6d558d7e400a4dd45edbf0941f5c WatchSource:0}: Error finding container 4819ed6e5da372be0f670059d3ee9c1d41dd6d558d7e400a4dd45edbf0941f5c: Status 404 returned error can't find the container with id 4819ed6e5da372be0f670059d3ee9c1d41dd6d558d7e400a4dd45edbf0941f5c
Jul 27 18:15:00.625856 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:15:00.625831    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ceaf7a_2245_457e_86b1_a7228a43a8ab.slice/crio-43cb0bc9ab242dd558e8f917ea75aa79839c02db2cab37e7738cda8b2bb7fdeb WatchSource:0}: Error finding container 43cb0bc9ab242dd558e8f917ea75aa79839c02db2cab37e7738cda8b2bb7fdeb: Status 404 returned error can't find the container with id 43cb0bc9ab242dd558e8f917ea75aa79839c02db2cab37e7738cda8b2bb7fdeb
Jul 27 18:15:01.030081 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.030045    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh" event={"ID":"66ceaf7a-2245-457e-86b1-a7228a43a8ab","Type":"ContainerStarted","Data":"e2fbe56f7704610e2c516a61cb453279f383e77308b3f4000ae40d2218140567"}
Jul 27 18:15:01.030236 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.030105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh" event={"ID":"66ceaf7a-2245-457e-86b1-a7228a43a8ab","Type":"ContainerStarted","Data":"43cb0bc9ab242dd558e8f917ea75aa79839c02db2cab37e7738cda8b2bb7fdeb"}
Jul 27 18:15:01.031407 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.031374    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc" event={"ID":"512d18a1-e88a-4a2c-b2d9-7b86647987b0","Type":"ContainerStarted","Data":"b984e5a39efe778f6e97e08de03da6863c4831b332e47224da658871071a1ed7"}
Jul 27 18:15:01.031735 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.031422    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc" event={"ID":"512d18a1-e88a-4a2c-b2d9-7b86647987b0","Type":"ContainerStarted","Data":"4819ed6e5da372be0f670059d3ee9c1d41dd6d558d7e400a4dd45edbf0941f5c"}
Jul 27 18:15:01.032862 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.032832    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b" event={"ID":"f131ae91-f29b-4ff9-97b0-06cb3c1ddb99","Type":"ContainerStarted","Data":"34240a62096f77245502fbe8d3241c88a527ef053c6c30247656291340eeed2b"}
Jul 27 18:15:01.032985 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.032877    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b" event={"ID":"f131ae91-f29b-4ff9-97b0-06cb3c1ddb99","Type":"ContainerStarted","Data":"8a2c68288dbc0f07d327c3f491cdf19ac59359e591061ed6986ac19e8ed2f342"}
Jul 27 18:15:01.034246 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.034224    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w" event={"ID":"fa897af7-a766-4d16-a951-860227c924be","Type":"ContainerStarted","Data":"3f79e3eb29d8e617c98bafb6b9bd92a506a1fe06c5f4129c42bf8623ceb88d69"}
Jul 27 18:15:01.034354 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.034271    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w" event={"ID":"fa897af7-a766-4d16-a951-860227c924be","Type":"ContainerStarted","Data":"8769b1b62eb803aacc294f4b6da2b8fde041159c55bebe697f1a6b9cbe071896"}
Jul 27 18:15:01.043747 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.043706    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh" podStartSLOduration=1.043693259 podStartE2EDuration="1.043693259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:15:01.042226094 +0000 UTC m=+3258.303201294" watchObservedRunningTime="2026-07-27 18:15:01.043693259 +0000 UTC m=+3258.304668457"
Jul 27 18:15:01.057385 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.057343    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc" podStartSLOduration=1.057331475 podStartE2EDuration="1.057331475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:15:01.055634353 +0000 UTC m=+3258.316609553" watchObservedRunningTime="2026-07-27 18:15:01.057331475 +0000 UTC m=+3258.318306674"
Jul 27 18:15:01.070124 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.070086    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b" podStartSLOduration=1.07007078 podStartE2EDuration="1.07007078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:15:01.069183281 +0000 UTC m=+3258.330158480" watchObservedRunningTime="2026-07-27 18:15:01.07007078 +0000 UTC m=+3258.331045980"
Jul 27 18:15:01.082855 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.082818    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w" podStartSLOduration=1.08280671 podStartE2EDuration="1.08280671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:15:01.082625577 +0000 UTC m=+3258.343600779" watchObservedRunningTime="2026-07-27 18:15:01.08280671 +0000 UTC m=+3258.343781909"
Jul 27 18:15:01.256691 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:01.256658    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fb6d227-df54-4854-a7b9-4eb93a79d56c" path="/var/lib/kubelet/pods/3fb6d227-df54-4854-a7b9-4eb93a79d56c/volumes"
Jul 27 18:15:02.045963 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.045844    2565 generic.go:356] "Generic (PLEG): container finished" podID="66ceaf7a-2245-457e-86b1-a7228a43a8ab" containerID="e2fbe56f7704610e2c516a61cb453279f383e77308b3f4000ae40d2218140567" exitCode=0
Jul 27 18:15:02.045963 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.045930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh" event={"ID":"66ceaf7a-2245-457e-86b1-a7228a43a8ab","Type":"ContainerDied","Data":"e2fbe56f7704610e2c516a61cb453279f383e77308b3f4000ae40d2218140567"}
Jul 27 18:15:02.047459 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.047436    2565 generic.go:356] "Generic (PLEG): container finished" podID="512d18a1-e88a-4a2c-b2d9-7b86647987b0" containerID="b984e5a39efe778f6e97e08de03da6863c4831b332e47224da658871071a1ed7" exitCode=0
Jul 27 18:15:02.047560 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.047499    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc" event={"ID":"512d18a1-e88a-4a2c-b2d9-7b86647987b0","Type":"ContainerDied","Data":"b984e5a39efe778f6e97e08de03da6863c4831b332e47224da658871071a1ed7"}
Jul 27 18:15:02.048911 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.048886    2565 generic.go:356] "Generic (PLEG): container finished" podID="f131ae91-f29b-4ff9-97b0-06cb3c1ddb99" containerID="34240a62096f77245502fbe8d3241c88a527ef053c6c30247656291340eeed2b" exitCode=0
Jul 27 18:15:02.049012 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.048977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b" event={"ID":"f131ae91-f29b-4ff9-97b0-06cb3c1ddb99","Type":"ContainerDied","Data":"34240a62096f77245502fbe8d3241c88a527ef053c6c30247656291340eeed2b"}
Jul 27 18:15:02.050338 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.050320    2565 generic.go:356] "Generic (PLEG): container finished" podID="fa897af7-a766-4d16-a951-860227c924be" containerID="3f79e3eb29d8e617c98bafb6b9bd92a506a1fe06c5f4129c42bf8623ceb88d69" exitCode=0
Jul 27 18:15:02.050388 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:02.050375    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w" event={"ID":"fa897af7-a766-4d16-a951-860227c924be","Type":"ContainerDied","Data":"3f79e3eb29d8e617c98bafb6b9bd92a506a1fe06c5f4129c42bf8623ceb88d69"}
Jul 27 18:15:03.193945 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:03.193922    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"
Jul 27 18:15:03.244756 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:03.244725    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"
Jul 27 18:15:03.274713 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:03.274691    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"
Jul 27 18:15:03.280050 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:03.280030    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b"
Jul 27 18:15:04.061868 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.061840    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"
Jul 27 18:15:04.061868 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.061849    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh" event={"ID":"66ceaf7a-2245-457e-86b1-a7228a43a8ab","Type":"ContainerDied","Data":"43cb0bc9ab242dd558e8f917ea75aa79839c02db2cab37e7738cda8b2bb7fdeb"}
Jul 27 18:15:04.062158 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.061901    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43cb0bc9ab242dd558e8f917ea75aa79839c02db2cab37e7738cda8b2bb7fdeb"
Jul 27 18:15:04.063108 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.063084    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc" event={"ID":"512d18a1-e88a-4a2c-b2d9-7b86647987b0","Type":"ContainerDied","Data":"4819ed6e5da372be0f670059d3ee9c1d41dd6d558d7e400a4dd45edbf0941f5c"}
Jul 27 18:15:04.063239 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.063121    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4819ed6e5da372be0f670059d3ee9c1d41dd6d558d7e400a4dd45edbf0941f5c"
Jul 27 18:15:04.063239 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.063090    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"
Jul 27 18:15:04.064442 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.064412    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b" event={"ID":"f131ae91-f29b-4ff9-97b0-06cb3c1ddb99","Type":"ContainerDied","Data":"8a2c68288dbc0f07d327c3f491cdf19ac59359e591061ed6986ac19e8ed2f342"}
Jul 27 18:15:04.064559 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.064446    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a2c68288dbc0f07d327c3f491cdf19ac59359e591061ed6986ac19e8ed2f342"
Jul 27 18:15:04.064559 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.064418    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752935-89g9b"
Jul 27 18:15:04.065978 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.065962    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"
Jul 27 18:15:04.066074 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.065990    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w" event={"ID":"fa897af7-a766-4d16-a951-860227c924be","Type":"ContainerDied","Data":"8769b1b62eb803aacc294f4b6da2b8fde041159c55bebe697f1a6b9cbe071896"}
Jul 27 18:15:04.066074 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:04.066029    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8769b1b62eb803aacc294f4b6da2b8fde041159c55bebe697f1a6b9cbe071896"
Jul 27 18:15:28.557821 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.557786    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"]
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558103    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa897af7-a766-4d16-a951-860227c924be" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558114    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa897af7-a766-4d16-a951-860227c924be" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558127    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ceaf7a-2245-457e-86b1-a7228a43a8ab" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558133    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ceaf7a-2245-457e-86b1-a7228a43a8ab" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558140    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="512d18a1-e88a-4a2c-b2d9-7b86647987b0" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558145    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="512d18a1-e88a-4a2c-b2d9-7b86647987b0" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558154    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f131ae91-f29b-4ff9-97b0-06cb3c1ddb99" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558160    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f131ae91-f29b-4ff9-97b0-06cb3c1ddb99" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558216    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="512d18a1-e88a-4a2c-b2d9-7b86647987b0" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558225    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa897af7-a766-4d16-a951-860227c924be" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558232    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f131ae91-f29b-4ff9-97b0-06cb3c1ddb99" containerName="cleanup"
Jul 27 18:15:28.558367 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.558238    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ceaf7a-2245-457e-86b1-a7228a43a8ab" containerName="cleanup"
Jul 27 18:15:28.563746 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.563714    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"
Jul 27 18:15:28.566413 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.566389    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 18:15:28.566413 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.566403    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:15:28.570799 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.570773    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"]
Jul 27 18:15:28.638875 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.638850    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7b0dbc-110c-4942-90e0-5c15c8b77f0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t\" (UID: \"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"
Jul 27 18:15:28.740103 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.740073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7b0dbc-110c-4942-90e0-5c15c8b77f0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t\" (UID: \"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"
Jul 27 18:15:28.740723 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.740706    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7b0dbc-110c-4942-90e0-5c15c8b77f0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t\" (UID: \"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"
Jul 27 18:15:28.877817 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:28.877735    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"
Jul 27 18:15:29.007489 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:29.007464    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"]
Jul 27 18:15:29.010180 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:15:29.010146    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7b0dbc_110c_4942_90e0_5c15c8b77f0d.slice/crio-c9ffe4e87d865e376ea954bcc6a465e1eae7acb973d510bfb500fdd1d52d0bf6 WatchSource:0}: Error finding container c9ffe4e87d865e376ea954bcc6a465e1eae7acb973d510bfb500fdd1d52d0bf6: Status 404 returned error can't find the container with id c9ffe4e87d865e376ea954bcc6a465e1eae7acb973d510bfb500fdd1d52d0bf6
Jul 27 18:15:29.187228 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:29.187147    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t" event={"ID":"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d","Type":"ContainerStarted","Data":"c9ffe4e87d865e376ea954bcc6a465e1eae7acb973d510bfb500fdd1d52d0bf6"}
Jul 27 18:15:30.192845 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:30.192813    2565 generic.go:356] "Generic (PLEG): container finished" podID="ef7b0dbc-110c-4942-90e0-5c15c8b77f0d" containerID="bfb6d996a8103587d1a70e27e71826996aebc8826a80bfdf09ac86ae534df158" exitCode=0
Jul 27 18:15:30.193244 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:30.192863    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t" event={"ID":"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d","Type":"ContainerDied","Data":"bfb6d996a8103587d1a70e27e71826996aebc8826a80bfdf09ac86ae534df158"}
Jul 27 18:15:31.328798 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:31.328774    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"
Jul 27 18:15:31.467815 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:31.467737    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7b0dbc-110c-4942-90e0-5c15c8b77f0d-maas-api-service-ca\") pod \"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d\" (UID: \"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d\") "
Jul 27 18:15:31.468096 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:31.468070    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef7b0dbc-110c-4942-90e0-5c15c8b77f0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef7b0dbc-110c-4942-90e0-5c15c8b77f0d" (UID: "ef7b0dbc-110c-4942-90e0-5c15c8b77f0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:15:31.568476 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:31.568449    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef7b0dbc-110c-4942-90e0-5c15c8b77f0d-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:15:32.204471 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:32.204434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t" event={"ID":"ef7b0dbc-110c-4942-90e0-5c15c8b77f0d","Type":"ContainerDied","Data":"c9ffe4e87d865e376ea954bcc6a465e1eae7acb973d510bfb500fdd1d52d0bf6"}
Jul 27 18:15:32.204648 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:32.204485    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9ffe4e87d865e376ea954bcc6a465e1eae7acb973d510bfb500fdd1d52d0bf6"
Jul 27 18:15:32.204648 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:32.204441    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"
Jul 27 18:15:39.045193 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:39.045160    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"]
Jul 27 18:15:39.046987 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:39.046967    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-fbef1a-29752935-5frnh"]
Jul 27 18:15:39.255456 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:39.255422    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66ceaf7a-2245-457e-86b1-a7228a43a8ab" path="/var/lib/kubelet/pods/66ceaf7a-2245-457e-86b1-a7228a43a8ab/volumes"
Jul 27 18:15:42.371651 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:42.371617    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:15:43.419186 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:43.419151    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:15:43.427761 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:43.427739    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:15:43.643157 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:43.643126    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"]
Jul 27 18:15:43.645559 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:43.645538    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-fbef1a-ppr9t"]
Jul 27 18:15:43.646840 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:43.646817    2565 scope.go:117] "RemoveContainer" containerID="7af7b5bcfc4b352295d0382fab1fae16929524c5551387831d018ebac2db9f4f"
Jul 27 18:15:45.261959 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:45.261926    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef7b0dbc-110c-4942-90e0-5c15c8b77f0d" path="/var/lib/kubelet/pods/ef7b0dbc-110c-4942-90e0-5c15c8b77f0d/volumes"
Jul 27 18:15:51.044239 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.044203    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"]
Jul 27 18:15:51.044743 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.044729    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef7b0dbc-110c-4942-90e0-5c15c8b77f0d" containerName="revoke-keys"
Jul 27 18:15:51.044835 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.044750    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef7b0dbc-110c-4942-90e0-5c15c8b77f0d" containerName="revoke-keys"
Jul 27 18:15:51.044895 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.044881    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef7b0dbc-110c-4942-90e0-5c15c8b77f0d" containerName="revoke-keys"
Jul 27 18:15:51.049187 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.049165    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"
Jul 27 18:15:51.051278 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.051253    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 18:15:51.051395 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.051378    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:15:51.055578 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.055554    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"]
Jul 27 18:15:51.124544 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.124516    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3999c023-4d78-4082-a61d-b97d327c2598-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5\" (UID: \"3999c023-4d78-4082-a61d-b97d327c2598\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"
Jul 27 18:15:51.225671 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.225628    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3999c023-4d78-4082-a61d-b97d327c2598-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5\" (UID: \"3999c023-4d78-4082-a61d-b97d327c2598\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"
Jul 27 18:15:51.226503 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.226471    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3999c023-4d78-4082-a61d-b97d327c2598-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5\" (UID: \"3999c023-4d78-4082-a61d-b97d327c2598\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"
Jul 27 18:15:51.360781 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.360744    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"
Jul 27 18:15:51.697803 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:51.697766    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"]
Jul 27 18:15:51.700317 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:15:51.700260    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3999c023_4d78_4082_a61d_b97d327c2598.slice/crio-400a19b5058c8bea29da1180ad545515b0216c18a1096ba3c31fdb51a1898a73 WatchSource:0}: Error finding container 400a19b5058c8bea29da1180ad545515b0216c18a1096ba3c31fdb51a1898a73: Status 404 returned error can't find the container with id 400a19b5058c8bea29da1180ad545515b0216c18a1096ba3c31fdb51a1898a73
Jul 27 18:15:52.300512 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:52.300472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5" event={"ID":"3999c023-4d78-4082-a61d-b97d327c2598","Type":"ContainerStarted","Data":"8978a638264b7e6a9a9f33c11245438d6bdc2fc4c2aeab18121ce933a40c932f"}
Jul 27 18:15:52.300889 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:52.300528    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5" event={"ID":"3999c023-4d78-4082-a61d-b97d327c2598","Type":"ContainerStarted","Data":"400a19b5058c8bea29da1180ad545515b0216c18a1096ba3c31fdb51a1898a73"}
Jul 27 18:15:52.316124 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:52.316077    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5" podStartSLOduration=1.316064031 podStartE2EDuration="1.316064031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:15:52.315140341 +0000 UTC m=+3309.576115542" watchObservedRunningTime="2026-07-27 18:15:52.316064031 +0000 UTC m=+3309.577039292"
Jul 27 18:15:53.307106 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:53.307069    2565 generic.go:356] "Generic (PLEG): container finished" podID="3999c023-4d78-4082-a61d-b97d327c2598" containerID="8978a638264b7e6a9a9f33c11245438d6bdc2fc4c2aeab18121ce933a40c932f" exitCode=0
Jul 27 18:15:53.307565 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:53.307154    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5" event={"ID":"3999c023-4d78-4082-a61d-b97d327c2598","Type":"ContainerDied","Data":"8978a638264b7e6a9a9f33c11245438d6bdc2fc4c2aeab18121ce933a40c932f"}
Jul 27 18:15:54.437707 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:54.437681    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"
Jul 27 18:15:54.456402 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:54.456379    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3999c023-4d78-4082-a61d-b97d327c2598-maas-api-service-ca\") pod \"3999c023-4d78-4082-a61d-b97d327c2598\" (UID: \"3999c023-4d78-4082-a61d-b97d327c2598\") "
Jul 27 18:15:54.456780 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:54.456744    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3999c023-4d78-4082-a61d-b97d327c2598-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3999c023-4d78-4082-a61d-b97d327c2598" (UID: "3999c023-4d78-4082-a61d-b97d327c2598"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:15:54.557633 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:54.557601    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3999c023-4d78-4082-a61d-b97d327c2598-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:15:55.319384 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:55.319353    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"
Jul 27 18:15:55.319564 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:55.319387    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5" event={"ID":"3999c023-4d78-4082-a61d-b97d327c2598","Type":"ContainerDied","Data":"400a19b5058c8bea29da1180ad545515b0216c18a1096ba3c31fdb51a1898a73"}
Jul 27 18:15:55.319564 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:15:55.319434    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="400a19b5058c8bea29da1180ad545515b0216c18a1096ba3c31fdb51a1898a73"
Jul 27 18:16:03.326794 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:03.326754    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"]
Jul 27 18:16:03.331905 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:03.331876    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1aa3ff-29752935-zznvc"]
Jul 27 18:16:05.256341 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:05.256235    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="512d18a1-e88a-4a2c-b2d9-7b86647987b0" path="/var/lib/kubelet/pods/512d18a1-e88a-4a2c-b2d9-7b86647987b0/volumes"
Jul 27 18:16:06.289136 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:06.289103    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"]
Jul 27 18:16:06.294516 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:06.294492    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1aa3ff-m7tb5"]
Jul 27 18:16:06.975242 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:06.975205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:16:07.259830 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:07.259752    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3999c023-4d78-4082-a61d-b97d327c2598" path="/var/lib/kubelet/pods/3999c023-4d78-4082-a61d-b97d327c2598/volumes"
Jul 27 18:16:13.639765 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.639732    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"]
Jul 27 18:16:13.640134 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.640054    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3999c023-4d78-4082-a61d-b97d327c2598" containerName="revoke-keys"
Jul 27 18:16:13.640134 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.640064    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3999c023-4d78-4082-a61d-b97d327c2598" containerName="revoke-keys"
Jul 27 18:16:13.640134 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.640129    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3999c023-4d78-4082-a61d-b97d327c2598" containerName="revoke-keys"
Jul 27 18:16:13.644179 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.644161    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"
Jul 27 18:16:13.646630 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.646603    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ldh7c\""
Jul 27 18:16:13.647125 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.647110    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 18:16:13.653727 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.653699    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"]
Jul 27 18:16:13.716861 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.716836    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c3c3e0-b410-4754-90d1-1ad4ed55b91e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q\" (UID: \"79c3c3e0-b410-4754-90d1-1ad4ed55b91e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"
Jul 27 18:16:13.817172 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.817144    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c3c3e0-b410-4754-90d1-1ad4ed55b91e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q\" (UID: \"79c3c3e0-b410-4754-90d1-1ad4ed55b91e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"
Jul 27 18:16:13.817757 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.817740    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c3c3e0-b410-4754-90d1-1ad4ed55b91e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q\" (UID: \"79c3c3e0-b410-4754-90d1-1ad4ed55b91e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"
Jul 27 18:16:13.955193 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:13.955104    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"
Jul 27 18:16:14.101709 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:14.101679    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"]
Jul 27 18:16:14.105163 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:16:14.105134    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c3c3e0_b410_4754_90d1_1ad4ed55b91e.slice/crio-7c033059984c7d9b31b2e27f5a9be9b06393f077fb2aaf48f9e484bdb6201117 WatchSource:0}: Error finding container 7c033059984c7d9b31b2e27f5a9be9b06393f077fb2aaf48f9e484bdb6201117: Status 404 returned error can't find the container with id 7c033059984c7d9b31b2e27f5a9be9b06393f077fb2aaf48f9e484bdb6201117
Jul 27 18:16:14.422182 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:14.422144    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q" event={"ID":"79c3c3e0-b410-4754-90d1-1ad4ed55b91e","Type":"ContainerStarted","Data":"bfee536b9303c4bd8d27e11ee42aaf3f6c8e109b1e38d1c6ffad3cfc99a8f861"}
Jul 27 18:16:14.422378 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:14.422207    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q" event={"ID":"79c3c3e0-b410-4754-90d1-1ad4ed55b91e","Type":"ContainerStarted","Data":"7c033059984c7d9b31b2e27f5a9be9b06393f077fb2aaf48f9e484bdb6201117"}
Jul 27 18:16:14.487095 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:14.487043    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q" podStartSLOduration=1.487026175 podStartE2EDuration="1.487026175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:16:14.486455241 +0000 UTC m=+3331.747430440" watchObservedRunningTime="2026-07-27 18:16:14.487026175 +0000 UTC m=+3331.748001373"
Jul 27 18:16:15.428926 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:15.428892    2565 generic.go:356] "Generic (PLEG): container finished" podID="79c3c3e0-b410-4754-90d1-1ad4ed55b91e" containerID="bfee536b9303c4bd8d27e11ee42aaf3f6c8e109b1e38d1c6ffad3cfc99a8f861" exitCode=0
Jul 27 18:16:15.429328 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:15.428977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q" event={"ID":"79c3c3e0-b410-4754-90d1-1ad4ed55b91e","Type":"ContainerDied","Data":"bfee536b9303c4bd8d27e11ee42aaf3f6c8e109b1e38d1c6ffad3cfc99a8f861"}
Jul 27 18:16:16.562942 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:16.562919    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"
Jul 27 18:16:16.643268 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:16.643239    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c3c3e0-b410-4754-90d1-1ad4ed55b91e-maas-api-service-ca\") pod \"79c3c3e0-b410-4754-90d1-1ad4ed55b91e\" (UID: \"79c3c3e0-b410-4754-90d1-1ad4ed55b91e\") "
Jul 27 18:16:16.643584 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:16.643563    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c3c3e0-b410-4754-90d1-1ad4ed55b91e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "79c3c3e0-b410-4754-90d1-1ad4ed55b91e" (UID: "79c3c3e0-b410-4754-90d1-1ad4ed55b91e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 18:16:16.744910 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:16.744827    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c3c3e0-b410-4754-90d1-1ad4ed55b91e-maas-api-service-ca\") on node \"ip-10-0-131-8.ec2.internal\" DevicePath \"\""
Jul 27 18:16:17.438885 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:17.438852    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q" event={"ID":"79c3c3e0-b410-4754-90d1-1ad4ed55b91e","Type":"ContainerDied","Data":"7c033059984c7d9b31b2e27f5a9be9b06393f077fb2aaf48f9e484bdb6201117"}
Jul 27 18:16:17.439054 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:17.438892    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"
Jul 27 18:16:17.439054 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:17.438900    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c033059984c7d9b31b2e27f5a9be9b06393f077fb2aaf48f9e484bdb6201117"
Jul 27 18:16:23.903106 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:23.903067    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"]
Jul 27 18:16:23.910965 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:23.910939    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ed5396-29752935-bck7w"]
Jul 27 18:16:25.255592 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:25.255559    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa897af7-a766-4d16-a951-860227c924be" path="/var/lib/kubelet/pods/fa897af7-a766-4d16-a951-860227c924be/volumes"
Jul 27 18:16:26.886469 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:26.886430    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pdhh9"]
Jul 27 18:16:28.737092 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:28.737059    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"]
Jul 27 18:16:28.739672 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:28.739649    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ed5396-jh95q"]
Jul 27 18:16:29.255454 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:29.255421    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c3c3e0-b410-4754-90d1-1ad4ed55b91e" path="/var/lib/kubelet/pods/79c3c3e0-b410-4754-90d1-1ad4ed55b91e/volumes"
Jul 27 18:16:36.169952 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:36.169915    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67dbb79db9-bqr6h_5111231d-9b81-4cb9-9f44-0799c34432e9/authorino/0.log"
Jul 27 18:16:41.739915 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:41.739871    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9kg5s_ed7a4ca3-159a-4f46-9d85-a512339877e5/server/0.log"
Jul 27 18:16:41.851253 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:41.851222    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pjkkx_821bb5e4-886a-4df1-8c7b-1399e7da2bb3/manager/0.log"
Jul 27 18:16:42.218377 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:42.218352    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-g8zdl_31466f32-8da3-4df6-a383-d777742f74fc/copy-manifests/0.log"
Jul 27 18:16:42.230203 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:42.230179    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-g8zdl_31466f32-8da3-4df6-a383-d777742f74fc/manager/0.log"
Jul 27 18:16:42.757691 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:42.757662    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29752920-vblmt_13561fba-5345-49ee-8c60-8fc0e066ec29/cleanup/0.log"
Jul 27 18:16:42.860548 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:42.860515    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29752935-89g9b_f131ae91-f29b-4ff9-97b0-06cb3c1ddb99/cleanup/0.log"
Jul 27 18:16:44.184956 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:44.184923    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67dbb79db9-bqr6h_5111231d-9b81-4cb9-9f44-0799c34432e9/authorino/0.log"
Jul 27 18:16:44.861196 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:44.861168    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pdhh9_efc57564-8fdc-45b9-8f97-82d3f6de4aeb/limitador/0.log"
Jul 27 18:16:45.548109 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:45.548078    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b8c977c58-q975s_00da9703-3511-4e59-8048-72e5cd010f14/kube-auth-proxy/0.log"
Jul 27 18:16:46.134810 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:46.134780    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58fd4bb687-9jltm_9d428abe-51d0-42d3-b610-0552dd60f476/router/0.log"
Jul 27 18:16:47.033732 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:47.033693    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-hhzcq_c6169d61-67d5-4c21-8fc9-e4638bee575d/main/0.log"
Jul 27 18:16:52.477204 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.477170    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dglvr/must-gather-7tkk9"]
Jul 27 18:16:52.477654 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.477546    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c3c3e0-b410-4754-90d1-1ad4ed55b91e" containerName="revoke-keys"
Jul 27 18:16:52.477654 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.477559    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c3c3e0-b410-4754-90d1-1ad4ed55b91e" containerName="revoke-keys"
Jul 27 18:16:52.477654 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.477631    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c3c3e0-b410-4754-90d1-1ad4ed55b91e" containerName="revoke-keys"
Jul 27 18:16:52.486414 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.486388    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.489334 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.489306    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dglvr\"/\"kube-root-ca.crt\""
Jul 27 18:16:52.489698 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.489680    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dglvr\"/\"openshift-service-ca.crt\""
Jul 27 18:16:52.489984 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.489966    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dglvr\"/\"default-dockercfg-s4v9h\""
Jul 27 18:16:52.494064 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.493925    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dglvr/must-gather-7tkk9"]
Jul 27 18:16:52.532353 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.532333    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lwpc\" (UniqueName: \"kubernetes.io/projected/d22b4e7d-c24a-4f00-9672-6c0ead839565-kube-api-access-7lwpc\") pod \"must-gather-7tkk9\" (UID: \"d22b4e7d-c24a-4f00-9672-6c0ead839565\") " pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.532465 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.532387    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d22b4e7d-c24a-4f00-9672-6c0ead839565-must-gather-output\") pod \"must-gather-7tkk9\" (UID: \"d22b4e7d-c24a-4f00-9672-6c0ead839565\") " pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.633149 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.633114    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d22b4e7d-c24a-4f00-9672-6c0ead839565-must-gather-output\") pod \"must-gather-7tkk9\" (UID: \"d22b4e7d-c24a-4f00-9672-6c0ead839565\") " pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.633288 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.633158    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lwpc\" (UniqueName: \"kubernetes.io/projected/d22b4e7d-c24a-4f00-9672-6c0ead839565-kube-api-access-7lwpc\") pod \"must-gather-7tkk9\" (UID: \"d22b4e7d-c24a-4f00-9672-6c0ead839565\") " pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.633459 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.633438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d22b4e7d-c24a-4f00-9672-6c0ead839565-must-gather-output\") pod \"must-gather-7tkk9\" (UID: \"d22b4e7d-c24a-4f00-9672-6c0ead839565\") " pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.640674 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.640645    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lwpc\" (UniqueName: \"kubernetes.io/projected/d22b4e7d-c24a-4f00-9672-6c0ead839565-kube-api-access-7lwpc\") pod \"must-gather-7tkk9\" (UID: \"d22b4e7d-c24a-4f00-9672-6c0ead839565\") " pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.797643 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.797566    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dglvr/must-gather-7tkk9"
Jul 27 18:16:52.936497 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.936472    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dglvr/must-gather-7tkk9"]
Jul 27 18:16:52.939028 ip-10-0-131-8 kubenswrapper[2565]: W0727 18:16:52.938987    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd22b4e7d_c24a_4f00_9672_6c0ead839565.slice/crio-0dd3f48b5602cc4da7f28cef5a344db52e0f254c684540a8be948515705be047 WatchSource:0}: Error finding container 0dd3f48b5602cc4da7f28cef5a344db52e0f254c684540a8be948515705be047: Status 404 returned error can't find the container with id 0dd3f48b5602cc4da7f28cef5a344db52e0f254c684540a8be948515705be047
Jul 27 18:16:52.940838 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:52.940819    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 18:16:53.621010 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:53.620977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dglvr/must-gather-7tkk9" event={"ID":"d22b4e7d-c24a-4f00-9672-6c0ead839565","Type":"ContainerStarted","Data":"0dd3f48b5602cc4da7f28cef5a344db52e0f254c684540a8be948515705be047"}
Jul 27 18:16:54.634339 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:54.634287    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dglvr/must-gather-7tkk9" event={"ID":"d22b4e7d-c24a-4f00-9672-6c0ead839565","Type":"ContainerStarted","Data":"fceb554ac7ad359562e992c757f2509bf032d9abe1eb45b883b1d85756af7328"}
Jul 27 18:16:55.641053 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:55.641018    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dglvr/must-gather-7tkk9" event={"ID":"d22b4e7d-c24a-4f00-9672-6c0ead839565","Type":"ContainerStarted","Data":"3711f59b6e3e5bf00dda1bad0ac6f039ca8335c1bd459e6368d7e8f0fe9fb754"}
Jul 27 18:16:55.657453 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:55.657403    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dglvr/must-gather-7tkk9" podStartSLOduration=2.113934144 podStartE2EDuration="3.65738761s" totalImagesPullingTime="1.543453466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:16:55.655838167 +0000 UTC m=+3372.916813389" watchObservedRunningTime="2026-07-27 18:16:55.65738761 +0000 UTC m=+3372.918362809"
Jul 27 18:16:56.058372 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:56.058339    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l288b_e4e307bf-7bb5-44dc-8dc0-890bbafffb78/global-pull-secret-syncer/0.log"
Jul 27 18:16:56.293195 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:56.293164    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vhdzk_60f24a69-e287-4fc4-a659-e0d275e17215/konnectivity-agent/0.log"
Jul 27 18:16:56.341160 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:16:56.341089    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-8.ec2.internal_a952e817f62df0bda7ddea87b49c2739/haproxy/0.log"
Jul 27 18:17:00.830310 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:00.829649    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67dbb79db9-bqr6h_5111231d-9b81-4cb9-9f44-0799c34432e9/authorino/0.log"
Jul 27 18:17:01.219582 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:01.219549    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pdhh9_efc57564-8fdc-45b9-8f97-82d3f6de4aeb/limitador/0.log"
Jul 27 18:17:03.139129 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:03.139087    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-947dp_92430eb1-2164-4d31-90dc-d80938eaacb7/cluster-monitoring-operator/0.log"
Jul 27 18:17:03.485203 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:03.485171    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xrd2m_b0c852db-9a78-43a4-8c96-18cac555b90e/node-exporter/0.log"
Jul 27 18:17:03.513679 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:03.513647    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xrd2m_b0c852db-9a78-43a4-8c96-18cac555b90e/kube-rbac-proxy/0.log"
Jul 27 18:17:03.541102 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:03.541049    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xrd2m_b0c852db-9a78-43a4-8c96-18cac555b90e/init-textfile/0.log"
Jul 27 18:17:03.961352 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:03.961323    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-657c6c8b9d-5lgw7_3da258f4-720e-4bc8-992f-2e73584c4948/telemeter-client/0.log"
Jul 27 18:17:03.995609 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:03.995583    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-657c6c8b9d-5lgw7_3da258f4-720e-4bc8-992f-2e73584c4948/reload/0.log"
Jul 27 18:17:04.014993 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:04.014966    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-657c6c8b9d-5lgw7_3da258f4-720e-4bc8-992f-2e73584c4948/kube-rbac-proxy/0.log"
Jul 27 18:17:05.474053 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:05.474017    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-pgb6x_76cab81e-4e12-45f8-b53c-2134434c8663/networking-console-plugin/0.log"
Jul 27 18:17:06.106061 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.106032    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/1.log"
Jul 27 18:17:06.113990 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.113963    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xjqb6_fb075801-f23d-4b07-a3e8-35b0dc2bda86/console-operator/2.log"
Jul 27 18:17:06.880969 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.880936    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"]
Jul 27 18:17:06.886664 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.886638    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:06.901105 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.901075    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"]
Jul 27 18:17:06.977071 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.977037    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-proc\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:06.977373 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.977338    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-lib-modules\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:06.977596 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.977477    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-sys\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:06.977596 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.977542    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb4zj\" (UniqueName: \"kubernetes.io/projected/5cf6b3e2-8bfd-4c22-8c75-28824f514347-kube-api-access-kb4zj\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:06.977736 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:06.977619    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-podres\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078427 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078394    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb4zj\" (UniqueName: \"kubernetes.io/projected/5cf6b3e2-8bfd-4c22-8c75-28824f514347-kube-api-access-kb4zj\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078427 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078436    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-podres\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078692 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078502    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-proc\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078692 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-lib-modules\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078692 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078631    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-proc\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078871 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078687    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-podres\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078871 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078733    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-lib-modules\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078871 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-sys\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.078871 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.078770    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cf6b3e2-8bfd-4c22-8c75-28824f514347-sys\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.088063 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.088035    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb4zj\" (UniqueName: \"kubernetes.io/projected/5cf6b3e2-8bfd-4c22-8c75-28824f514347-kube-api-access-kb4zj\") pod \"perf-node-gather-daemonset-m42xv\" (UID: \"5cf6b3e2-8bfd-4c22-8c75-28824f514347\") " pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.198344 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.198239    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:07.375602 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.375040    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"]
Jul 27 18:17:07.432995 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.432973    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-fmm85_123a82b2-e990-49bd-897a-4628586d8e49/volume-data-source-validator/0.log"
Jul 27 18:17:07.757847 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.757813    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv" event={"ID":"5cf6b3e2-8bfd-4c22-8c75-28824f514347","Type":"ContainerStarted","Data":"56df06077cbc5635d828dae4f9662a8ae1d736fb03715365b9f0ae02a48ea6c4"}
Jul 27 18:17:07.758058 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.757875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv" event={"ID":"5cf6b3e2-8bfd-4c22-8c75-28824f514347","Type":"ContainerStarted","Data":"53069b065e759a4450315bbc5c2b901b697a78dda9ebc7ee9bea5e473cd70fdd"}
Jul 27 18:17:07.758058 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:07.758025    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:08.724724 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:08.724697    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xxzzn_ee9b370c-7951-4a22-89bc-596ef49a2ad4/dns/0.log"
Jul 27 18:17:08.749830 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:08.749807    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xxzzn_ee9b370c-7951-4a22-89bc-596ef49a2ad4/kube-rbac-proxy/0.log"
Jul 27 18:17:08.779676 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:08.779652    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2kb9q_4f91c41b-a2d4-4635-840f-09bf2302fd91/dns-node-resolver/0.log"
Jul 27 18:17:09.432606 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:09.432573    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c7rx4_bfe4dd64-9586-428a-a704-c3d5cec3b3e7/node-ca/0.log"
Jul 27 18:17:10.700870 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:10.700837    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b8c977c58-q975s_00da9703-3511-4e59-8048-72e5cd010f14/kube-auth-proxy/0.log"
Jul 27 18:17:10.942629 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:10.942597    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58fd4bb687-9jltm_9d428abe-51d0-42d3-b610-0552dd60f476/router/0.log"
Jul 27 18:17:11.771338 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:11.771279    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7r696_29645a34-e258-423f-acbd-aa709a57ffd7/serve-healthcheck-canary/0.log"
Jul 27 18:17:12.278350 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:12.278324    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-mqz4z_76f6fe48-c826-4af3-bc3d-ab70010dcb93/insights-operator/0.log"
Jul 27 18:17:12.278515 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:12.278323    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-mqz4z_76f6fe48-c826-4af3-bc3d-ab70010dcb93/insights-operator/1.log"
Jul 27 18:17:12.300548 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:12.300522    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8zj7b_830ccf32-fb7c-461b-bf99-4aa55b3ad9a5/kube-rbac-proxy/0.log"
Jul 27 18:17:12.324112 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:12.324083    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8zj7b_830ccf32-fb7c-461b-bf99-4aa55b3ad9a5/exporter/0.log"
Jul 27 18:17:12.348424 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:12.348395    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8zj7b_830ccf32-fb7c-461b-bf99-4aa55b3ad9a5/extractor/0.log"
Jul 27 18:17:13.773932 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:13.773903    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv"
Jul 27 18:17:13.789844 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:13.789791    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dglvr/perf-node-gather-daemonset-m42xv" podStartSLOduration=7.789776295 podStartE2EDuration="7.789776295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 18:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:17:07.781352834 +0000 UTC m=+3385.042328034" watchObservedRunningTime="2026-07-27 18:17:13.789776295 +0000 UTC m=+3391.050751493"
Jul 27 18:17:15.383808 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:15.383779    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9kg5s_ed7a4ca3-159a-4f46-9d85-a512339877e5/server/0.log"
Jul 27 18:17:15.484666 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:15.484623    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pjkkx_821bb5e4-886a-4df1-8c7b-1399e7da2bb3/manager/0.log"
Jul 27 18:17:15.700696 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:15.700666    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-g8zdl_31466f32-8da3-4df6-a383-d777742f74fc/manager/0.log"
Jul 27 18:17:15.720505 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:15.720476    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-746b8d4d47-g8zdl_31466f32-8da3-4df6-a383-d777742f74fc/copy-manifests/0.log"
Jul 27 18:17:16.701727 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:16.701694    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b8c977c58-q975s_00da9703-3511-4e59-8048-72e5cd010f14/kube-auth-proxy/0.log"
Jul 27 18:17:16.887659 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:16.887619    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58fd4bb687-9jltm_9d428abe-51d0-42d3-b610-0552dd60f476/router/0.log"
Jul 27 18:17:17.212815 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:17.212727    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76849bb8b4-wdrzp_943dd6cb-3abc-4725-b364-af87695556bb/manager/0.log"
Jul 27 18:17:18.500127 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:18.500100    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b8c977c58-q975s_00da9703-3511-4e59-8048-72e5cd010f14/kube-auth-proxy/0.log"
Jul 27 18:17:18.692599 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:18.692526    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58fd4bb687-9jltm_9d428abe-51d0-42d3-b610-0552dd60f476/router/0.log"
Jul 27 18:17:22.321000 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:22.320976    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-5wsx6_7c73813b-830d-4f2a-8163-49d6a971270a/migrator/0.log"
Jul 27 18:17:22.343018 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:22.342986    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-5wsx6_7c73813b-830d-4f2a-8163-49d6a971270a/graceful-termination/0.log"
Jul 27 18:17:22.701719 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:22.701639    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9_1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be/kube-storage-version-migrator-operator/1.log"
Jul 27 18:17:22.702677 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:22.702651    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-gj9f9_1ce5495a-aaf3-4fd5-81cd-4ba5751dd6be/kube-storage-version-migrator-operator/0.log"
Jul 27 18:17:23.744398 ip-10-0-131-8 kubenswrapper[2565]: I0727 18:17:23.744329    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-525fp_3408bd3e-f7d9-43a3-a172-f8c55e43e578/kube-multus/0.log"